diff options
| author | Jason DeTiberus <jdetiber@redhat.com> | 2015-04-01 15:09:19 -0400 | 
|---|---|---|
| committer | Jason DeTiberus <jdetiber@redhat.com> | 2015-04-14 23:29:16 -0400 | 
| commit | 6a4b7a5eb6c4b5e747bab795e2428d7c3992f559 (patch) | |
| tree | 2519948f1eb8c372192ed4fd8805adc71da8433d | |
| parent | c85e91fdca031eba06481a24f74aa076ae9a4d38 (diff) | |
| download | openshift-6a4b7a5eb6c4b5e747bab795e2428d7c3992f559.tar.gz openshift-6a4b7a5eb6c4b5e747bab795e2428d7c3992f559.tar.bz2 openshift-6a4b7a5eb6c4b5e747bab795e2428d7c3992f559.tar.xz openshift-6a4b7a5eb6c4b5e747bab795e2428d7c3992f559.zip | |
Configuration updates for latest builds and major refactor
Configuration updates for latest builds
- Switch to using create-node-config
- Switch sdn services to use etcd over SSL
- This re-uses the client certificate deployed on each node
- Additional node registration changes
- Do not assume that metadata service is available in openshift_facts module
- Call systemctl daemon-reload after installing openshift-master, openshift-sdn-master, openshift-node, openshift-sdn-node
- Fix bug overriding openshift_hostname and openshift_public_hostname in byo playbooks
- Start moving generated configs to /etc/openshift
- Some custom module cleanup
- Add known issue with ansible-1.9 to README_OSE.md
- Update to genericize the kubernetes_register_node module
  - Default to use kubectl for commands
  - Allow for overriding kubectl_cmd
  - In openshift_register_node role, override kubectl_cmd to openshift_kube
- Set default openshift_registry_url for enterprise when deployment_type is enterprise
- Fix openshift_register_node for client config change
- Ensure that master certs directory is created
- Add roles and filter_plugin symlinks to playbooks/common/openshift-master and node
- Allow non-root user with sudo nopasswd access
- Updates for README_OSE.md
- Update byo inventory for adding additional comments
- Updates for node cert/config sync to work with non-root user using sudo
- Move node config/certs to /etc/openshift/node
- Don't use path for mktemp. addresses: https://github.com/openshift/openshift-ansible/issues/154
Create common playbooks
- create common/openshift-master/config.yml
- create common/openshift-node/config.yml
- update playbooks to use new common playbooks
- update launch playbooks to call update playbooks
- fix openshift_registry and openshift_node_ip usage
Set default deployment type to origin
- openshift_repo updates for enabling origin deployments
  - also separate repo and gpgkey file structure
  - remove kubernetes repo since it isn't currently needed
- full deployment type support for bin/cluster
  - honor OS_DEPLOYMENT_TYPE env variable
  - add --deployment-type option, which will override OS_DEPLOYMENT_TYPE if set
  - if neither OS_DEPLOYMENT_TYPE or --deployment-type is set, defaults to
    origin installs
Additional changes:
- Add separate config action to bin/cluster that runs ansible config but does
  not update packages
- Some more duplication reduction in cluster playbooks.
- Rename task files in playbooks dirs to have tasks in their name for clarity.
- update aws/gce scripts to use a directory for inventory (otherwise when
  there are no hosts returned from dynamic inventory there is an error)
libvirt refactor and update
- add libvirt dynamic inventory
- updates to use dynamic inventory for libvirt
111 files changed, 1686 insertions, 1198 deletions
| diff --git a/README_OSE.md b/README_OSE.md index 6ebdb7f99..6d4a9ba92 100644 --- a/README_OSE.md +++ b/README_OSE.md @@ -7,15 +7,17 @@  * [Creating the default variables for the hosts and host groups](#creating-the-default-variables-for-the-hosts-and-host-groups)  * [Running the ansible playbooks](#running-the-ansible-playbooks)  * [Post-ansible steps](#post-ansible-steps) +* [Overriding detected ip addresses and hostnames](#overriding-detected-ip-addresses-and-hostnames)  ## Requirements  * ansible -  * Tested using ansible-1.8.2-1.fc20.noarch, but should work with version 1.8+ +  * Tested using ansible-1.8.4-1.fc20.noarch, but should work with version 1.8+ +  * There is currently a known issue with ansible-1.9.0, you can downgrade to 1.8.4 on Fedora by installing one of the bulids from Koji: http://koji.fedoraproject.org/koji/packageinfo?packageID=13842    * Available in Fedora channels    * Available for EL with EPEL and Optional channel  * One or more RHEL 7.1 VMs -* ssh key based auth for the root user needs to be pre-configured from the host -  running ansible to the remote hosts +* Either ssh key based auth for the root user or ssh key based auth for a user +  with sudo access (no password)  * A checkout of openshift-ansible from https://github.com/openshift/openshift-ansible/    ```sh @@ -48,9 +50,6 @@ subscription-manager repos \  ```  * Configuration of router is not automated yet  * Configuration of docker-registry is not automated yet -* End-to-end testing has not been completed yet using this module -* root user is used for all ansible actions; eventually we will support using -  a non-root user with sudo.  ## Configuring the host inventory  [Ansible docs](http://docs.ansible.com/intro_inventory.html) @@ -64,6 +63,38 @@ option to ansible-playbook.  ```ini  # This is an example of a bring your own (byo) host inventory +# Create an OSEv3 group that contains the maters and nodes groups +[OSEv3:children] +masters +nodes + +# Set variables common for all OSEv3 hosts +[OSEv3:vars] +# SSH user, this user should allow ssh based auth without requiring a password +ansible_ssh_user=root + +# If ansible_ssh_user is not root, ansible_sudo must be set to true +#ansible_sudo=true + +# To deploy origin, change deployment_type to origin +deployment_type=enterprise + +# Pre-release registry URL +openshift_registry_url=docker-buildvm-rhose.usersys.redhat.com:5000/openshift3_beta/ose-${component}:${version} + +# Pre-release additional repo +openshift_additional_repos=[{'id': 'ose-devel', 'name': 'ose-devel', +'baseurl': +'http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/3.0/latest/RH7-RHOSE-3.0/$basearch/os', +'enabled': 1, 'gpgcheck': 0}] + +# Origin copr repo +#openshift_additional_repos=[{'id': 'openshift-origin-copr', 'name': +'OpenShift Origin COPR', 'baseurl': +'https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/epel-7-$basearch/', +'enabled': 1, 'gpgcheck': 1, gpgkey: +'https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/pubkey.gpg'}] +  # host group for masters  [masters]  ose3-master.example.com @@ -76,51 +107,13 @@ ose3-node[1:2].example.com  The hostnames above should resolve both from the hosts themselves and  the host where ansible is running (if different). -## Creating the default variables for the hosts and host groups -[Ansible docs](http://docs.ansible.com/intro_inventory.html#id9) - -#### Group vars for all hosts -/etc/ansible/group_vars/all: -```yaml ---- -# Assume that we want to use the root as the ssh user for all hosts -ansible_ssh_user: root - -# Default debug level for all OpenShift hosts -openshift_debug_level: 4 - -# Set the OpenShift deployment type for all hosts -openshift_deployment_type: enterprise - -# Override the default registry for development -openshift_registry_url: docker-buildvm-rhose.usersys.redhat.com:5000/openshift3_beta/ose-${component}:${version} - -# To use the latest OpenShift Enterprise Errata puddle: -#openshift_additional_repos: -#- id: ose-devel -#  name: ose-devel -#  baseurl: http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterpriseErrata/3.0/latest/RH7-RHOSE-3.0/$basearch/os -#  enabled: 1 -#  gpgcheck: 0 -# To use the latest OpenShift Enterprise Whitelist puddle: -openshift_additional_repos: -- id: ose-devel -  name: ose-devel -  baseurl: http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/3.0/latest/RH7-RHOSE-3.0/$basearch/os -  enabled: 1 -  gpgcheck: 0 - -``` -  ## Running the ansible playbooks  From the openshift-ansible checkout run:  ```sh  ansible-playbook playbooks/byo/config.yml  ``` -**Note:** this assumes that the host inventory is /etc/ansible/hosts and the -group_vars are defined in /etc/ansible/group_vars, if using a different -inventory file (and a group_vars directory that is in the same directory as -the directory as the inventory) use the -i option for ansible-playbook. +**Note:** this assumes that the host inventory is /etc/ansible/hosts, if using a different +inventory file use the -i option for ansible-playbook.  ## Post-ansible steps  #### Create the default router @@ -140,3 +133,109 @@ openshift ex registry --create=true \    --images='docker-buildvm-rhose.usersys.redhat.com:5000/openshift3_beta/ose-${component}:${version}' \    --mount-host=/var/lib/openshift/docker-registry  ``` + +## Overriding detected ip addresses and hostnames +Some deployments will require that the user override the detected hostnames +and ip addresses for the hosts. To see what the default values will be you can +run the openshift_facts playbook: +```sh +ansible-playbook playbooks/byo/openshift_facts.yml +``` +The output will be similar to: +``` +ok: [10.3.9.45] => { +    "result": { +        "ansible_facts": { +            "openshift": { +                "common": { +                    "hostname": "jdetiber-osev3-ansible-005dcfa6-27c6-463d-9b95-ef059579befd.os1.phx2.redhat.com", +                    "ip": "172.16.4.79", +                    "public_hostname": "jdetiber-osev3-ansible-005dcfa6-27c6-463d-9b95-ef059579befd.os1.phx2.redhat.com", +                    "public_ip": "10.3.9.45", +                    "use_openshift_sdn": true +                }, +                "provider": { +                  ... <snip> ... +                } +            } +        }, +        "changed": false, +        "invocation": { +            "module_args": "", +            "module_name": "openshift_facts" +        } +    } +} +ok: [10.3.9.42] => { +    "result": { +        "ansible_facts": { +            "openshift": { +                "common": { +                    "hostname": "jdetiber-osev3-ansible-c6ae8cdc-ba0b-4a81-bb37-14549893f9d3.os1.phx2.redhat.com", +                    "ip": "172.16.4.75", +                    "public_hostname": "jdetiber-osev3-ansible-c6ae8cdc-ba0b-4a81-bb37-14549893f9d3.os1.phx2.redhat.com", +                    "public_ip": "10.3.9.42", +                    "use_openshift_sdn": true +                }, +                "provider": { +                  ...<snip>... +                } +            } +        }, +        "changed": false, +        "invocation": { +            "module_args": "", +            "module_name": "openshift_facts" +        } +    } +} +ok: [10.3.9.36] => { +    "result": { +        "ansible_facts": { +            "openshift": { +                "common": { +                    "hostname": "jdetiber-osev3-ansible-bc39a3d3-cdd7-42fe-9c12-9fac9b0ec320.os1.phx2.redhat.com", +                    "ip": "172.16.4.73", +                    "public_hostname": "jdetiber-osev3-ansible-bc39a3d3-cdd7-42fe-9c12-9fac9b0ec320.os1.phx2.redhat.com", +                    "public_ip": "10.3.9.36", +                    "use_openshift_sdn": true +                }, +                "provider": { +                    ...<snip>... +                } +            } +        }, +        "changed": false, +        "invocation": { +            "module_args": "", +            "module_name": "openshift_facts" +        } +    } +} +``` +Now, we want to verify the detected common settings to verify that they are +what we expect them to be (if not, we can override them). + +* hostname +  * Should resolve to the internal ip from the instances themselves. +  * openshift_hostname will override. +* ip +  * Should be the internal ip of the instance. +  * openshift_ip will override. +* public hostname +  * Should resolve to the external ip from hosts outside of the cloud +  * provider openshift_public_hostname will override. +* public_ip +  * Should be the externally accessible ip associated with the instance +  * openshift_public_ip will override +* use_openshift_sdn +  * Should be true unless the cloud is GCE. +  * openshift_use_openshift_sdn overrides + +To override the the defaults, you can set the variables in your inventory: +``` +...snip... +[masters] +ose3-master.example.com openshift_ip=1.1.1.1 openshift_hostname=ose3-master.example.com openshift_public_ip=2.2.2.2 openshift_public_hostname=ose3-master.public.example.com +...snip... +``` diff --git a/README_libvirt.md b/README_libvirt.md index fd2eb57f6..bcbaf4bd5 100644 --- a/README_libvirt.md +++ b/README_libvirt.md @@ -1,4 +1,3 @@ -  LIBVIRT Setup instructions  ========================== @@ -9,19 +8,21 @@ This makes `libvirt` useful to develop, test and debug Openshift and openshift-a  Install dependencies  -------------------- -1. Install [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) -2. Install [ebtables](http://ebtables.netfilter.org/) -3. Install [qemu](http://wiki.qemu.org/Main_Page) -4. Install [libvirt](http://libvirt.org/) -5. Enable and start the libvirt daemon, e.g: -   * ``systemctl enable libvirtd`` -   * ``systemctl start libvirtd`` -6. [Grant libvirt access to your user¹](https://libvirt.org/aclpolkit.html) -7. Check that your `$HOME` is accessible to the qemu user² +1.	Install [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) +2.	Install [ebtables](http://ebtables.netfilter.org/) +3.	Install [qemu](http://wiki.qemu.org/Main_Page) +4.	Install [libvirt](http://libvirt.org/) +5.	Enable and start the libvirt daemon, e.g: +	-	`systemctl enable libvirtd` +	-	`systemctl start libvirtd` +6.	[Grant libvirt access to your user¹](https://libvirt.org/aclpolkit.html) +7.	Check that your `$HOME` is accessible to the qemu user² +8.	Configure dns resolution on the host³  #### ¹ Depending on your distribution, libvirt access may be denied by default or may require a password at each access.  You can test it with the following command: +  ```  virsh -c qemu:///system pool-list  ``` @@ -67,12 +68,7 @@ If your `$HOME` is world readable, everything is fine. If your `$HOME` is privat  error: Cannot access storage file '$HOME/libvirt-storage-pool-openshift/lenaic-master-216d8.qcow2' (as uid:99, gid:78): Permission denied  ``` -In order to fix that issue, you have several possibilities: -* set `libvirt_storage_pool_path` inside `playbooks/libvirt/openshift-cluster/launch.yml` and `playbooks/libvirt/openshift-cluster/terminate.yml` to a directory: -  * backed by a filesystem with a lot of free disk space -  * writable by your user; -  * accessible by the qemu user. -* Grant the qemu user access to the storage pool. +In order to fix that issue, you have several possibilities:* set `libvirt_storage_pool_path` inside `playbooks/libvirt/openshift-cluster/launch.yml` and `playbooks/libvirt/openshift-cluster/terminate.yml` to a directory: * backed by a filesystem with a lot of free disk space * writable by your user; * accessible by the qemu user.* Grant the qemu user access to the storage pool.  On Arch: @@ -80,13 +76,55 @@ On Arch:  setfacl -m g:kvm:--x ~  ``` -Test the setup +#### ³ Enabling DNS resolution to your guest VMs with NetworkManager + +-	Verify NetworkManager is configured to use dnsmasq: + +```sh +$ sudo vi /etc/NetworkManager/NetworkManager.conf +[main] +dns=dnsmasq +``` + +-	Configure dnsmasq to use the Virtual Network router for example.com: + +```sh +sudo vi /etc/NetworkManager/dnsmasq.d/libvirt_dnsmasq.conf server=/example.com/192.168.55.1 +``` + +Test The Setup  -------------- +1.	cd openshift-ansible/ +2.	Try to list all instances (Passing an empty string as the cluster_id argument will result in all libvirt instances being listed) + +``` +  bin/cluster list libvirt ''  ``` -cd openshift-ansible -bin/cluster create -m 1 -n 3 libvirt lenaic +Creating a cluster +------------------ + +1.	To create a cluster with one master and two nodes -bin/cluster terminate libvirt lenaic +``` +  bin/cluster create libvirt lenaic +``` + +Updating a cluster +------------------ + +1.	To update the cluster + +``` +  bin/cluster update libvirt lenaic +``` + +Terminating a cluster +--------------------- + +1.	To terminate the cluster + +``` +  bin/cluster terminate libvirt lenaic  ``` diff --git a/bin/cluster b/bin/cluster index ca227721e..79f1f988f 100755 --- a/bin/cluster +++ b/bin/cluster @@ -22,13 +22,28 @@ class Cluster(object):                  '-o ControlPersist=600s '              ) +    def get_deployment_type(self, args): +        """ +        Get the deployment_type based on the environment variables and the +        command line arguments +        :param args: command line arguments provided by the user +        :return: string representing the deployment type +        """ +        deployment_type = 'origin' +        if args.deployment_type: +            deployment_type = args.deployment_type +        elif 'OS_DEPLOYMENT_TYPE' in os.environ: +            deployment_type = os.environ['OS_DEPLOYMENT_TYPE'] +        return deployment_type +      def create(self, args):          """          Create an OpenShift cluster for given provider          :param args: command line arguments provided by user          :return: exit status from run command          """ -        env = {'cluster_id': args.cluster_id} +        env = {'cluster_id': args.cluster_id, +               'deployment_type': self.get_deployment_type(args)}          playbook = "playbooks/{}/openshift-cluster/launch.yml".format(args.provider)          inventory = self.setup_provider(args.provider) @@ -43,7 +58,8 @@ class Cluster(object):          :param args: command line arguments provided by user          :return: exit status from run command          """ -        env = {'cluster_id': args.cluster_id} +        env = {'cluster_id': args.cluster_id, +               'deployment_type': self.get_deployment_type(args)}          playbook = "playbooks/{}/openshift-cluster/terminate.yml".format(args.provider)          inventory = self.setup_provider(args.provider) @@ -55,19 +71,34 @@ class Cluster(object):          :param args: command line arguments provided by user          :return: exit status from run command          """ -        env = {'cluster_id': args.cluster_id} +        env = {'cluster_id': args.cluster_id, +               'deployment_type': self.get_deployment_type(args)}          playbook = "playbooks/{}/openshift-cluster/list.yml".format(args.provider)          inventory = self.setup_provider(args.provider)          return self.action(args, inventory, env, playbook) +    def config(self, args): +        """ +        Configure or reconfigure OpenShift across clustered VMs +        :param args: command line arguments provided by user +        :return: exit status from run command +        """ +        env = {'cluster_id': args.cluster_id, +               'deployment_type': self.get_deployment_type(args)} +        playbook = "playbooks/{}/openshift-cluster/config.yml".format(args.provider) +        inventory = self.setup_provider(args.provider) + +        return self.action(args, inventory, env, playbook) +      def update(self, args):          """          Update to latest OpenShift across clustered VMs          :param args: command line arguments provided by user          :return: exit status from run command          """ -        env = {'cluster_id': args.cluster_id} +        env = {'cluster_id': args.cluster_id, +               'deployment_type': self.get_deployment_type(args)}          playbook = "playbooks/{}/openshift-cluster/update.yml".format(args.provider)          inventory = self.setup_provider(args.provider) @@ -81,19 +112,19 @@ class Cluster(object):          """          config = ConfigParser.ConfigParser()          if 'gce' == provider: -            config.readfp(open('inventory/gce/gce.ini')) +            config.readfp(open('inventory/gce/hosts/gce.ini'))              for key in config.options('gce'):                  os.environ[key] = config.get('gce', key) -            inventory = '-i inventory/gce/gce.py' +            inventory = '-i inventory/gce/hosts'          elif 'aws' == provider: -            config.readfp(open('inventory/aws/ec2.ini')) +            config.readfp(open('inventory/aws/hosts/ec2.ini'))              for key in config.options('ec2'):                  os.environ[key] = config.get('ec2', key) -            inventory = '-i inventory/aws/ec2.py' +            inventory = '-i inventory/aws/hosts'          elif 'libvirt' == provider:              inventory = '-i inventory/libvirt/hosts'          else: @@ -145,29 +176,49 @@ if __name__ == '__main__':      parser = argparse.ArgumentParser(          description='Python wrapper to ensure proper environment for OpenShift ansible playbooks',      ) -    parser.add_argument('-v', '--verbose', action='count', help='Multiple -v options increase the verbosity') +    parser.add_argument('-v', '--verbose', action='count', +                        help='Multiple -v options increase the verbosity')      parser.add_argument('--version', action='version', version='%(prog)s 0.2')      meta_parser = argparse.ArgumentParser(add_help=False)      meta_parser.add_argument('provider', choices=providers, help='provider')      meta_parser.add_argument('cluster_id', help='prefix for cluster VM names') - -    action_parser = parser.add_subparsers(dest='action', title='actions', description='Choose from valid actions') - -    create_parser = action_parser.add_parser('create', help='Create a cluster', parents=[meta_parser]) -    create_parser.add_argument('-m', '--masters', default=1, type=int, help='number of masters to create in cluster') -    create_parser.add_argument('-n', '--nodes', default=2, type=int, help='number of nodes to create in cluster') +    meta_parser.add_argument('-t', '--deployment-type', +                             choices=['origin', 'online', 'enterprise'], +                             help='Deployment type. (default: origin)') + +    action_parser = parser.add_subparsers(dest='action', title='actions', +                                          description='Choose from valid actions') + +    create_parser = action_parser.add_parser('create', help='Create a cluster', +                                             parents=[meta_parser]) +    create_parser.add_argument('-m', '--masters', default=1, type=int, +                               help='number of masters to create in cluster') +    create_parser.add_argument('-n', '--nodes', default=2, type=int, +                               help='number of nodes to create in cluster')      create_parser.set_defaults(func=cluster.create) -    terminate_parser = action_parser.add_parser('terminate', help='Destroy a cluster', parents=[meta_parser]) -    terminate_parser.add_argument('-f', '--force', action='store_true', help='Destroy cluster without confirmation') +    config_parser = action_parser.add_parser('config', +                                             help='Configure or reconfigure a cluster', +                                             parents=[meta_parser]) +    config_parser.set_defaults(func=cluster.config) + +    terminate_parser = action_parser.add_parser('terminate', +                                                help='Destroy a cluster', +                                                parents=[meta_parser]) +    terminate_parser.add_argument('-f', '--force', action='store_true', +                                  help='Destroy cluster without confirmation')      terminate_parser.set_defaults(func=cluster.terminate) -    update_parser = action_parser.add_parser('update', help='Update OpenShift across cluster', parents=[meta_parser]) -    update_parser.add_argument('-f', '--force', action='store_true', help='Update cluster without confirmation') +    update_parser = action_parser.add_parser('update', +                                             help='Update OpenShift across cluster', +                                             parents=[meta_parser]) +    update_parser.add_argument('-f', '--force', action='store_true', +                               help='Update cluster without confirmation')      update_parser.set_defaults(func=cluster.update) -    list_parser = action_parser.add_parser('list', help='List VMs in cluster', parents=[meta_parser]) +    list_parser = action_parser.add_parser('list', help='List VMs in cluster', +                                           parents=[meta_parser])      list_parser.set_defaults(func=cluster.list)      args = parser.parse_args() diff --git a/filter_plugins/oo_filters.py b/filter_plugins/oo_filters.py index 1cf02218c..cf30cde9a 100644 --- a/filter_plugins/oo_filters.py +++ b/filter_plugins/oo_filters.py @@ -5,6 +5,7 @@  from ansible import errors, runner  import json  import pdb +import re  def oo_pdb(arg):      ''' This pops you into a pdb instance where arg is the data passed in from the filter. @@ -101,6 +102,18 @@ def oo_prepend_strings_in_list(data, prepend):      retval = [prepend + s for s in data]      return retval +def oo_get_deployment_type_from_groups(data): +    ''' This takes a list of groups and returns the associated +        deployment-type +    ''' +    if not issubclass(type(data), list): +        raise errors.AnsibleFilterError("|failed expects first param is a list") +    regexp = re.compile('^tag_deployment-type[-_]') +    matches = filter(regexp.match, data) +    if len(matches) > 0: +        return regexp.sub('', matches[0]) +    return "Unknown" +  class FilterModule (object):      def filters(self):          return { @@ -109,5 +122,6 @@ class FilterModule (object):                  "oo_flatten": oo_flatten,                  "oo_len": oo_len,                  "oo_pdb": oo_pdb, -                "oo_prepend_strings_in_list": oo_prepend_strings_in_list +                "oo_prepend_strings_in_list": oo_prepend_strings_in_list, +                "oo_get_deployment_type_from_groups": oo_get_deployment_type_from_groups                  } diff --git a/inventory/aws/group_vars/all b/inventory/aws/group_vars/all deleted file mode 100644 index b22da00de..000000000 --- a/inventory/aws/group_vars/all +++ /dev/null @@ -1,2 +0,0 @@ ---- -ansible_ssh_user: root diff --git a/inventory/aws/ec2.ini b/inventory/aws/hosts/ec2.ini index eaab0a410..eaab0a410 100644 --- a/inventory/aws/ec2.ini +++ b/inventory/aws/hosts/ec2.ini diff --git a/inventory/aws/ec2.py b/inventory/aws/hosts/ec2.py index f231ff4c2..f231ff4c2 100755 --- a/inventory/aws/ec2.py +++ b/inventory/aws/hosts/ec2.py diff --git a/inventory/aws/hosts/hosts b/inventory/aws/hosts/hosts new file mode 100644 index 000000000..6c590ac93 --- /dev/null +++ b/inventory/aws/hosts/hosts @@ -0,0 +1 @@ +localhost ansible_sudo=no ansible_python_interpreter=/usr/bin/python2 diff --git a/inventory/byo/group_vars/all b/inventory/byo/group_vars/all deleted file mode 100644 index d63e96668..000000000 --- a/inventory/byo/group_vars/all +++ /dev/null @@ -1,28 +0,0 @@ ---- -# lets assume that we want to use the root as the ssh user for all hosts -ansible_ssh_user: root - -# default debug level for all OpenShift hosts -openshift_debug_level: 4 - -# set the OpenShift deployment type for all hosts -openshift_deployment_type: enterprise - -# Override the default registry for development -openshift_registry_url: docker-buildvm-rhose.usersys.redhat.com:5000/openshift3_beta/ose-${component}:${version} - -# Use latest Errata puddle as an additional repo: -#openshift_additional_repos: -#- id: ose-devel -#  name: ose-devel -#  baseurl: http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterpriseErrata/3.0/latest/RH7-RHOSE-3.0/$basearch/os -#  enabled: 1 -#  gpgcheck: 0 - -# Use latest Whitelist puddle as an additional repo: -openshift_additional_repos: -- id: ose-devel -  name: ose-devel -  baseurl: http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/3.0/latest/RH7-RHOSE-3.0/$basearch/os -  enabled: 1 -  gpgcheck: 0 diff --git a/inventory/byo/hosts b/inventory/byo/hosts index 2dd854778..e9af5e571 100644 --- a/inventory/byo/hosts +++ b/inventory/byo/hosts @@ -1,5 +1,30 @@  # This is an example of a bring your own (byo) host inventory +# Create an OSEv3 group that contains the maters and nodes groups +[OSEv3:children] +masters +nodes + +# Set variables common for all OSEv3 hosts +[OSEv3:vars] +# SSH user, this user should allow ssh based auth without requiring a password +ansible_ssh_user=root + +# If ansible_ssh_user is not root, ansible_sudo must be set to true +#ansible_sudo=true + +# To deploy origin, change deployment_type to origin +deployment_type=enterprise + +# Pre-release registry URL +openshift_registry_url=docker-buildvm-rhose.usersys.redhat.com:5000/openshift3_beta/ose-${component}:${version} + +# Pre-release additional repo +openshift_additional_repos=[{'id': 'ose-devel', 'name': 'ose-devel', 'baseurl': 'http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/3.0/latest/RH7-RHOSE-3.0/$basearch/os', 'enabled': 1, 'gpgcheck': 0}] + +# Origin copr repo +#openshift_additional_repos=[{'id': 'openshift-origin-copr', 'name': 'OpenShift Origin COPR', 'baseurl': 'https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/epel-7-$basearch/', 'enabled': 1, 'gpgcheck': 1, gpgkey: 'https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/pubkey.gpg'}] +  # host group for masters  [masters]  ose3-master-ansible.test.example.com @@ -7,4 +32,3 @@ ose3-master-ansible.test.example.com  # host group for nodes  [nodes]  ose3-node[1:2]-ansible.test.example.com - diff --git a/inventory/gce/group_vars/all b/inventory/gce/group_vars/all deleted file mode 100644 index b22da00de..000000000 --- a/inventory/gce/group_vars/all +++ /dev/null @@ -1,2 +0,0 @@ ---- -ansible_ssh_user: root diff --git a/inventory/gce/gce.py b/inventory/gce/hosts/gce.py index 3403f735e..3403f735e 100755 --- a/inventory/gce/gce.py +++ b/inventory/gce/hosts/gce.py diff --git a/inventory/gce/hosts/hosts b/inventory/gce/hosts/hosts new file mode 100644 index 000000000..6c590ac93 --- /dev/null +++ b/inventory/gce/hosts/hosts @@ -0,0 +1 @@ +localhost ansible_sudo=no ansible_python_interpreter=/usr/bin/python2 diff --git a/inventory/libvirt/group_vars/all b/inventory/libvirt/group_vars/all deleted file mode 100644 index b22da00de..000000000 --- a/inventory/libvirt/group_vars/all +++ /dev/null @@ -1,2 +0,0 @@ ---- -ansible_ssh_user: root diff --git a/inventory/libvirt/hosts b/inventory/libvirt/hosts deleted file mode 100644 index 6a818f268..000000000 --- a/inventory/libvirt/hosts +++ /dev/null @@ -1,2 +0,0 @@ -# Eventually we'll add the GCE, AWS, etc dynamic inventories, but for now... -localhost ansible_python_interpreter=/usr/bin/python2 diff --git a/inventory/libvirt/hosts/hosts b/inventory/libvirt/hosts/hosts new file mode 100644 index 000000000..9cdc31449 --- /dev/null +++ b/inventory/libvirt/hosts/hosts @@ -0,0 +1 @@ +localhost ansible_sudo=no ansible_python_interpreter=/usr/bin/python2 connection=local diff --git a/inventory/libvirt/hosts/libvirt.ini b/inventory/libvirt/hosts/libvirt.ini new file mode 100644 index 000000000..62ff204dd --- /dev/null +++ b/inventory/libvirt/hosts/libvirt.ini @@ -0,0 +1,20 @@ +# Ansible libvirt external inventory script settings +# + +[libvirt] + +uri = qemu:///system + +# API calls to libvirt can be slow. For this reason, we cache the results of an API +# call. Set this to the path you want cache files to be written to. Two files +# will be written to this directory: +#   - ansible-libvirt.cache +#   - ansible-libvirt.index +cache_path = /tmp + +# The number of seconds a cache file is considered valid. After this many +# seconds, a new API call will be made, and the cache file will be updated. +cache_max_age = 900 + + + diff --git a/inventory/libvirt/hosts/libvirt_generic.py b/inventory/libvirt/hosts/libvirt_generic.py new file mode 100755 index 000000000..0a98e2af3 --- /dev/null +++ b/inventory/libvirt/hosts/libvirt_generic.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python + +""" +libvirt external inventory script +================================= + +Ansible has a feature where instead of reading from /etc/ansible/hosts +as a text file, it can query external programs to obtain the list +of hosts, groups the hosts are in, and even variables to assign to each host. + +To use this, copy this file over /etc/ansible/hosts and chmod +x the file. +This, more or less, allows you to keep one central database containing +info about all of your managed instances. + +""" + +# (c) 2015, Jason DeTiberus <jdetiber@redhat.com> +# +# This file is part of Ansible, +# +# Ansible is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# Ansible is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Ansible.  If not, see <http://www.gnu.org/licenses/>. + +###################################################################### + +import argparse +import ConfigParser +import os +import re +import sys +from time import time +import libvirt +import xml.etree.ElementTree as ET + +try: +    import json +except ImportError: +    import simplejson as json + + +class LibvirtInventory(object): + +    def __init__(self): +        self.inventory = dict()  # A list of groups and the hosts in that group +        self.cache = dict()  # Details about hosts in the inventory + +        # Read settings and parse CLI arguments +        self.read_settings() +        self.parse_cli_args() + +        if self.args.host: +            print self.json_format_dict(self.get_host_info(), self.args.pretty) +        elif self.args.list: +            print self.json_format_dict(self.get_inventory(), self.args.pretty) +        else:  # default action with no options +            print self.json_format_dict(self.get_inventory(), self.args.pretty) + +    def read_settings(self): +        config = ConfigParser.SafeConfigParser() +        config.read( +            os.path.dirname(os.path.realpath(__file__)) + '/libvirt.ini' +        ) +        self.libvirt_uri = config.get('libvirt', 'uri') + +    def parse_cli_args(self): +        parser = argparse.ArgumentParser( +            description='Produce an Ansible Inventory file based on libvirt' +        ) +        parser.add_argument( +            '--list', +            action='store_true', +            default=True, +            help='List instances (default: True)' +        ) +        parser.add_argument( +            '--host', +            action='store', +            help='Get all the variables about a specific instance' +        ) +        parser.add_argument( +            '--pretty', +            action='store_true', +            default=False, +            help='Pretty format (default: False)' +        ) +        self.args = parser.parse_args() + +    def get_host_info(self): +        inventory = self.get_inventory() +        if self.args.host in inventory['_meta']['hostvars']: +            return inventory['_meta']['hostvars'][self.args.host] + +    def get_inventory(self): +        inventory = dict(_meta=dict(hostvars=dict())) + +        conn = libvirt.openReadOnly(self.libvirt_uri) +        if conn is None: +            print "Failed to open connection to %s" % libvirt_uri +            sys.exit(1) + +        domains = conn.listAllDomains() +        if domains is None: +            print "Failed to list domains for connection %s" % libvirt_uri +            sys.exit(1) + +        arp_entries = self.parse_arp_entries() + +        for domain in domains: +            hostvars = dict(libvirt_name=domain.name(), +                            libvirt_id=domain.ID(), +                            libvirt_uuid=domain.UUIDString()) +            domain_name = domain.name() + +            # TODO: add support for guests that are not in a running state +            state, _ = domain.state() +            # 2 is the state for a running guest +            if state != 1: +                continue + +            hostvars['libvirt_status'] = 'running' + +            root = ET.fromstring(domain.XMLDesc()) +            ns = {'ansible': 'https://github.com/ansible/ansible'} +            for tag_elem in root.findall('./metadata/ansible:tag', ns): +                tag = tag_elem.text +                self.push(inventory, "tag_%s" % tag, domain_name) +                self.push(hostvars, 'libvirt_tags', tag) + +            # TODO: support more than one network interface, also support +            # interface types other than 'network' +            interface = root.find("./devices/interface[@type='network']") +            if interface is not None: +                mac_elem = interface.find('mac') +                if mac_elem is not None: +                    mac = mac_elem.get('address') +                    if mac in arp_entries: +                        ip_address = arp_entries[mac]['ip_address'] +                        hostvars['ansible_ssh_host'] = ip_address +                        hostvars['libvirt_ip_address'] = ip_address + +            inventory['_meta']['hostvars'][domain_name] = hostvars + +        return inventory + +    def parse_arp_entries(self): +        arp_entries = dict() +        with open('/proc/net/arp', 'r') as f: +            # throw away the header +            f.readline() + +            for line in f: +                ip_address, _, _, mac, _, device = line.strip().split() +                arp_entries[mac] = dict(ip_address=ip_address, device=device) + +        return arp_entries + +    def push(self, my_dict, key, element): +        if key in my_dict: +            my_dict[key].append(element) +        else: +            my_dict[key] = [element] + +    def json_format_dict(self, data, pretty=False): +        if pretty: +            return json.dumps(data, sort_keys=True, indent=2) +        else: +            return json.dumps(data) + +LibvirtInventory() diff --git a/playbooks/aws/openshift-cluster/config.yml b/playbooks/aws/openshift-cluster/config.yml new file mode 100644 index 000000000..b8961704e --- /dev/null +++ b/playbooks/aws/openshift-cluster/config.yml @@ -0,0 +1,36 @@ +--- +- name: Populate oo_masters_to_config host group +  hosts: localhost +  gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - name: Evaluate oo_masters_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_masters_to_config +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type_{{ cluster_id }}-openshift-master"] | default([]) +  - name: Evaluate oo_nodes_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_config +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type_{{ cluster_id }}-openshift-node"] | default([]) +  - name: Evaluate oo_first_master +    add_host: +      name: "{{ groups['tag_env-host-type_' ~ cluster_id ~ '-openshift-master'][0] }}" +      groups: oo_first_master +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    when: "'tag_env-host-type_{{ cluster_id }}-openshift-master' in groups" + +- include: ../../common/openshift-cluster/config.yml +  vars: +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}" +    openshift_hostname: "{{ ec2_private_ip_address }}" +    openshift_public_hostname: "{{ ec2_ip_address }}" diff --git a/playbooks/aws/openshift-cluster/launch.yml b/playbooks/aws/openshift-cluster/launch.yml index 3561c1803..e7125ea0c 100644 --- a/playbooks/aws/openshift-cluster/launch.yml +++ b/playbooks/aws/openshift-cluster/launch.yml @@ -4,59 +4,26 @@    connection: local    gather_facts: no    vars_files: -      - vars.yml +  - vars.yml    tasks: -    - set_fact: k8s_type="master" - -    - name: Generate master instance names(s) -      set_fact: scratch={{ cluster_id }}-{{ k8s_type }}-{{ '%05x' |format( 1048576 |random) }} -      register: master_names_output -      with_sequence: start=1 end={{ num_masters }} - -    # These set_fact's cannot be combined -    - set_fact: -        master_names_string: "{% for item in master_names_output.results %}{{ item.ansible_facts.scratch }} {% endfor %}" - -    - set_fact: -        master_names: "{{ master_names_string.strip().split(' ') }}" - -    - include: launch_instances.yml -      vars: -        instances: "{{ master_names }}" -        cluster: "{{ cluster_id }}" -        type: "{{ k8s_type }}" - -    - set_fact: k8s_type="node" - -    - name: Generate node instance names(s) -      set_fact: scratch={{ cluster_id }}-{{ k8s_type }}-{{ '%05x' |format( 1048576 |random) }} -      register: node_names_output -      with_sequence: start=1 end={{ num_nodes }} - -    # These set_fact's cannot be combined -    - set_fact: -        node_names_string: "{% for item in node_names_output.results %}{{ item.ansible_facts.scratch }} {% endfor %}" - -    - set_fact: -        node_names: "{{ node_names_string.strip().split(' ') }}" - -    - include: launch_instances.yml -      vars: -        instances: "{{ node_names }}" -        cluster: "{{ cluster_id }}" -        type: "{{ k8s_type }}" - -- hosts: "tag_env_{{ cluster_id }}" -  roles: -  - openshift_repos -  - os_update_latest - -- include: ../openshift-master/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type_{{ cluster_id }}-openshift-master\"]" - -- include: ../openshift-node/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type_{{ cluster_id }}-openshift-node\"]" +  - fail: +      msg: Deployment type not supported for libvirt provider yet +    when: deployment_type == 'enterprise' + +  - include: ../../common/openshift-cluster/set_master_launch_facts_tasks.yml +  - include: tasks/launch_instances.yml +    vars: +      instances: "{{ master_names }}" +      cluster: "{{ cluster_id }}" +      type: "{{ k8s_type }}" + +  - include: ../../common/openshift-cluster/set_node_launch_facts_tasks.yml +  - include: tasks/launch_instances.yml +    vars: +      instances: "{{ node_names }}" +      cluster: "{{ cluster_id }}" +      type: "{{ k8s_type }}" + +- include: update.yml  - include: list.yml diff --git a/playbooks/aws/openshift-cluster/list.yml b/playbooks/aws/openshift-cluster/list.yml index 08e9e2df4..5c04bc320 100644 --- a/playbooks/aws/openshift-cluster/list.yml +++ b/playbooks/aws/openshift-cluster/list.yml @@ -2,16 +2,23 @@  - name: Generate oo_list_hosts group    hosts: localhost    gather_facts: no +  vars_files: +  - vars.yml    tasks:    - set_fact: scratch_group=tag_env_{{ cluster_id }}      when: cluster_id != ''    - set_fact: scratch_group=all -    when: scratch_group is not defined -  - add_host: name={{ item }} groups=oo_list_hosts -    with_items: groups[scratch_group] | difference(['localhost']) +    when: cluster_id == '' +  - add_host: +      name: "{{ item }}" +      groups: oo_list_hosts +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost'])  - name: List Hosts    hosts: oo_list_hosts    gather_facts: no    tasks: -  - debug: msg="public:{{hostvars[inventory_hostname].ec2_ip_address}} private:{{hostvars[inventory_hostname].ec2_private_ip_address}}" +  - debug: +      msg: "public ip:{{ hostvars[inventory_hostname].ec2_ip_address }} private ip:{{ hostvars[inventory_hostname].ec2_private_ip_address }} deployment-type: {{ hostvars[inventory_hostname].group_names | oo_get_deployment_type_from_groups }}" diff --git a/playbooks/aws/openshift-cluster/launch_instances.yml b/playbooks/aws/openshift-cluster/tasks/launch_instances.yml index 9d645fbe5..58b4082df 100644 --- a/playbooks/aws/openshift-cluster/launch_instances.yml +++ b/playbooks/aws/openshift-cluster/tasks/launch_instances.yml @@ -1,8 +1,9 @@  --- +# TODO: modify machine_image based on deployment_type  - set_fact: -    machine_type: "{{ lookup('env', 'ec2_instance_type')|default('m3.large', true) }}" -    machine_image: "{{ lookup('env', 'ec2_ami')|default('ami-307b3658', true) }}" -    machine_region: "{{ lookup('env', 'ec2_region')|default('us-east-1', true) }}" +    machine_type: "{{ lookup('env', 'ec2_instance_type') | default('m3.large', true) }}" +    machine_image: "{{ lookup('env', 'ec2_ami') | default(deployment_vars[deployment_type].image, true) }}" +    machine_region: "{{ lookup('env', 'ec2_region') | default(deployment_vars[deployment_type].region, true) }}"      machine_keypair: "{{ lookup('env', 'ec2_keypair')|default('libra', true) }}"      created_by: "{{ lookup('env', 'LOGNAME')|default(cluster, true) }}"      security_group: "{{ lookup('env', 'ec2_security_group')|default('public', true) }}" @@ -25,6 +26,7 @@        env: "{{ env }}"        host-type: "{{ host_type }}"        env-host-type: "{{ env_host_type }}" +      deployment-type: "{{ deployment_type }}"    register: ec2  - name: Add Name tag to instances @@ -37,12 +39,14 @@        Name: "{{ item.0 }}"  - set_fact: -    instance_groups: tag_created-by_{{ created_by }}, tag_env_{{ env }}, tag_host-type_{{ host_type }}, tag_env-host-type_{{ env_host_type }} +    instance_groups: tag_created-by_{{ created_by }}, tag_env_{{ env }}, tag_host-type_{{ host_type }}, tag_env-host-type_{{ env_host_type }}, tag_deployment-type_{{ deployment_type }}  - name: Add new instances groups and variables    add_host:      hostname: "{{ item.0 }}"      ansible_ssh_host: "{{ item.1.dns_name }}" +    ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +    ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}"      groups: "{{ instance_groups }}"      ec2_private_ip_address: "{{ item.1.private_ip }}"      ec2_ip_address: "{{ item.1.public_ip }}" @@ -54,10 +58,12 @@    wait_for: "port=22 host={{ item.dns_name }}"    with_items: ec2.instances -- name: Wait for root user setup -  command: "ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null root@{{ item.dns_name }} echo root user is setup" +- name: Wait for user setup +  command: "ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null {{ hostvars[item.0].ansible_ssh_user }}@{{ item.1.dns_name }} echo {{ hostvars[item.0].ansible_ssh_user }} user is setup"    register: result    until: result.rc == 0    retries: 20    delay: 10 -  with_items: ec2.instances +  with_together: +  - instances +  - ec2.instances diff --git a/playbooks/aws/openshift-cluster/terminate.yml b/playbooks/aws/openshift-cluster/terminate.yml index 39607633a..1d2b60594 100644 --- a/playbooks/aws/openshift-cluster/terminate.yml +++ b/playbooks/aws/openshift-cluster/terminate.yml @@ -1,14 +1,26 @@  ---  - name: Terminate instance(s)    hosts: localhost - +  gather_facts: no    vars_files: -    - vars.yml +  - vars.yml +  tasks: +  - set_fact: scratch_group=tag_env-host-type_{{ cluster_id }}-openshift-node +  - add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_terminate +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost']) + +  - set_fact: scratch_group=tag_env-host-type_{{ cluster_id }}-openshift-master +  - add_host: +      name: "{{ item }}" +      groups: oo_masters_to_terminate +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost'])  - include: ../openshift-node/terminate.yml -  vars: -    oo_host_group_exp: 'groups["tag_env-host-type_{{ cluster_id }}-openshift-node"]'  - include: ../openshift-master/terminate.yml -  vars: -    oo_host_group_exp: 'groups["tag_env-host-type_{{ cluster_id }}-openshift-master"]' diff --git a/playbooks/aws/openshift-cluster/update.yml b/playbooks/aws/openshift-cluster/update.yml index 90ecdc6ab..5e7ab4e58 100644 --- a/playbooks/aws/openshift-cluster/update.yml +++ b/playbooks/aws/openshift-cluster/update.yml @@ -1,13 +1,18 @@  --- -- hosts: "tag_env_{{ cluster_id }}" -  roles: -  - openshift_repos -  - os_update_latest +- name: Populate oo_hosts_to_update group +  hosts: localhost +  gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - name: Evaluate oo_hosts_to_update +    add_host: +      name: "{{ item }}" +      groups: oo_hosts_to_update +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type_{{ cluster_id }}-openshift-master"] | union(groups["tag_env-host-type_{{ cluster_id }}-openshift-node"]) | default([]) -- include: ../openshift-master/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type_{{ cluster_id }}-openshift-master\"]" +- include: ../../common/openshift-cluster/update_repos_and_packages.yml -- include: ../openshift-node/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type_{{ cluster_id }}-openshift-node\"]" +- include: config.yml diff --git a/playbooks/aws/openshift-cluster/vars.yml b/playbooks/aws/openshift-cluster/vars.yml index ed97d539c..f0df3d6f5 100644 --- a/playbooks/aws/openshift-cluster/vars.yml +++ b/playbooks/aws/openshift-cluster/vars.yml @@ -1 +1,20 @@  --- +deployment_vars: +  origin: +    # fedora, since centos requires marketplace +    image: ami-acd999c4 +    region: us-east-1 +    ssh_user: fedora +    sudo: yes +  online: +    # private ami +    image: ami-307b3658 +    region: us-east-1 +    ssh_user: root +    sudo: no +  enterprise: +    # rhel-7.1, requires cloud access subscription +    image: ami-10663b78 +    region: us-east-1 +    ssh_user: ec2-user +    sudo: yes diff --git a/playbooks/aws/openshift-master/config.yml b/playbooks/aws/openshift-master/config.yml index 1c4060eee..37ab4fbe6 100644 --- a/playbooks/aws/openshift-master/config.yml +++ b/playbooks/aws/openshift-master/config.yml @@ -1,24 +1,19 @@  --- -- name: Populate oo_masters_to_config host group if needed +- name: Populate oo_masters_to_config host group    hosts: localhost    gather_facts: no    tasks: -  - name: "Evaluate oo_host_group_exp if it's set" -    add_host: "name={{ item }} groups=oo_masters_to_config" -    with_items: "{{ oo_host_group_exp | default('') }}" -    when: oo_host_group_exp is defined +  - name: Evaluate oo_masters_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_masters_to_config +      ansible_ssh_user: root +    with_items: oo_host_group_exp | default([]) -- name: Configure instances -  hosts: oo_masters_to_config +- include: ../../common/openshift-master/config.yml    vars: +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}"      openshift_hostname: "{{ ec2_private_ip_address }}"      openshift_public_hostname: "{{ ec2_ip_address }}" -    # TODO: this should be removed once openshift-sdn packages are available -    openshift_use_openshift_sdn: False -  vars_files: -  - vars.yml -  roles: -    - openshift_master -    #- openshift_sdn_master -    - pods -    - os_env_extras diff --git a/playbooks/aws/openshift-master/launch.yml b/playbooks/aws/openshift-master/launch.yml index 3d87879a0..6b3751682 100644 --- a/playbooks/aws/openshift-master/launch.yml +++ b/playbooks/aws/openshift-master/launch.yml @@ -4,14 +4,12 @@    connection: local    gather_facts: no +# TODO: modify atomic_ami based on deployment_type    vars:      inst_region: us-east-1      atomic_ami: ami-86781fee      user_data_file: user_data.txt -  vars_files: -    - vars.yml -    tasks:      - name: Launch instances        ec2: @@ -40,7 +38,7 @@            Name: "{{ item.0 }}"      - name: Add other tags to instances -      ec2_tag: "resource={{ item.id }} region={{ inst_region }} state=present" +      ec2_tag: resource={{ item.id }} region={{ inst_region }} state=present        with_items: ec2.instances        args:          tags: "{{ oo_new_inst_tags }}" @@ -57,7 +55,7 @@          - ec2.instances      - name: Wait for ssh -      wait_for: "port=22 host={{ item.dns_name }}" +      wait_for: port=22 host={{ item.dns_name }}        with_items: ec2.instances      - name: Wait for root user setup diff --git a/playbooks/aws/openshift-master/terminate.yml b/playbooks/aws/openshift-master/terminate.yml index fd15cf00f..a790336b1 100644 --- a/playbooks/aws/openshift-master/terminate.yml +++ b/playbooks/aws/openshift-master/terminate.yml @@ -1,15 +1,15 @@  --- -- name: Populate oo_masters_to_terminate host group if needed +- name: Populate oo_masters_to_terminate host group    hosts: localhost    gather_facts: no    tasks: -    - name: Evaluate oo_host_group_exp if it's set -      add_host: "name={{ item }} groups=oo_masters_to_terminate" -      with_items: "{{ oo_host_group_exp | default('') }}" -      when: oo_host_group_exp is defined +    - name: Evaluate oo_masters_to_terminate +      add_host: name={{ item }} groups=oo_masters_to_terminate +      with_items: oo_host_group_exp | default([]) -- name: Gather facts for instances to terminate +- name: Gather dynamic inventory variables for hosts to terminate    hosts: oo_masters_to_terminate +  gather_facts: no  - name: Terminate instances    hosts: localhost @@ -27,11 +27,12 @@        ignore_errors: yes        register: ec2_term        with_items: host_vars +      when: "'oo_masters_to_terminate' in groups"      # Fail if any of the instances failed to terminate with an error other      # than 403 Forbidden      - fail: msg=Terminating instance {{ item.item.ec2_id }} failed with message {{ item.msg }} -      when: "item.failed and not item.msg | search(\"error: EC2ResponseError: 403 Forbidden\")" +      when: "'oo_masters_to_terminate' in groups and item.failed and not item.msg | search(\"error: EC2ResponseError: 403 Forbidden\")"        with_items: ec2_term.results      - name: Stop instance if termination failed @@ -42,6 +43,7 @@        register: ec2_stop        when: item.failed        with_items: ec2_term.results +      when: "'oo_masters_to_terminate' in groups"      - name: Rename stopped instances        ec2_tag: resource={{ item.item.item.ec2_id }} region={{ item.item.item.ec2_region }} state=present @@ -49,4 +51,5 @@          tags:            Name: "{{ item.item.item.ec2_tag_Name }}-terminate"        with_items: ec2_stop.results +      when: "'oo_masters_to_terminate' in groups" diff --git a/playbooks/aws/openshift-master/vars.yml b/playbooks/aws/openshift-master/vars.yml deleted file mode 100644 index c196b2fca..000000000 --- a/playbooks/aws/openshift-master/vars.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -openshift_debug_level: 4 -openshift_cluster_id: "{{ cluster_id }}" diff --git a/playbooks/aws/openshift-node/config.yml b/playbooks/aws/openshift-node/config.yml index b08ed7571..fc9b397b4 100644 --- a/playbooks/aws/openshift-node/config.yml +++ b/playbooks/aws/openshift-node/config.yml @@ -1,107 +1,25 @@  --- -- name: Populate oo_nodes_to_config host group if needed +- name: Populate oo_nodes_to_config and oo_first_master host groups    hosts: localhost    gather_facts: no    tasks: -  - name: Evaluate oo_host_group_exp -    add_host: "name={{ item }} groups=oo_nodes_to_config" -    with_items: "{{ oo_host_group_exp | default('') }}" -    when: oo_host_group_exp is defined -  - add_host: +  - name: Evaluate oo_nodes_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_config +      ansible_ssh_user: root +    with_items: oo_host_group_exp | default([]) +  - name: Evaluate oo_first_master +    add_host:        name: "{{ groups['tag_env-host-type_' ~ cluster_id ~ '-openshift-master'][0] }}"        groups: oo_first_master -    when: oo_host_group_exp is defined +      ansible_ssh_user: root -- name: Gather and set facts for hosts to configure -  hosts: oo_nodes_to_config -  roles: -  - openshift_facts -  tasks: -  # Since the master is registering the nodes before they are configured, we -  # need to make sure to set the node properties beforehand if we do not want -  # the defaults -  - openshift_facts: -      role: "{{ item.role }}" -      local_facts: "{{ item.local_facts }}" -    with_items: -    - role: common -      local_facts: -        hostname: "{{ ec2_private_ip_address }}" -        public_hostname: "{{ ec2_ip_address }}" -        # TODO: this should be removed once openshift-sdn packages are available -        use_openshift_sdn: False -    - role: node -      local_facts: -        external_id: "{{ openshift_node_external_id | default(None) }}" -        resources_cpu: "{{ openshfit_node_resources_cpu | default(None) }}" -        resources_memory: "{{ openshfit_node_resources_memory | default(None) }}" -        pod_cidr: "{{ openshfit_node_pod_cidr | default(None) }}" -        labels: "{{ openshfit_node_labels | default(None) }}" -        annotations: "{{ openshfit_node_annotations | default(None) }}" - - -- name: Register nodes -  hosts: oo_first_master -  vars: -    openshift_nodes: "{{ hostvars -          | oo_select_keys(groups['oo_nodes_to_config']) }}" -  roles: -  - openshift_register_nodes -  tasks: -  - name: Create local temp directory for syncing certs -    local_action: command /usr/bin/mktemp -d /tmp/openshift-ansible-XXXXXXX -    register: mktemp - -  - name: Sync master certs to localhost -    synchronize: -      mode: pull -      checksum: yes -      src: /var/lib/openshift/openshift.local.certificates -      dest: "{{ mktemp.stdout }}" - - -- name: Configure instances -  hosts: oo_nodes_to_config -  vars_files: -  - vars.yml +- include: ../../common/openshift-node/config.yml    vars: +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}"      openshift_hostname: "{{ ec2_private_ip_address }}"      openshift_public_hostname: "{{ ec2_ip_address }}" -    sync_tmpdir: "{{ hostvars[groups['oo_first_master'][0]].mktemp.stdout }}" -    cert_parent_rel_path: openshift.local.certificates -    cert_rel_path: "{{ cert_parent_rel_path }}/node-{{ openshift.common.hostname }}" -    cert_base_path: /var/lib/openshift -    cert_parent_path: "{{ cert_base_path }}/{{ cert_parent_rel_path }}" -    cert_path: "{{ cert_base_path }}/{{ cert_rel_path }}" -  pre_tasks: -  - name: Ensure certificate directories exists -    file: -      path: "{{ item }}" -      state: directory -    with_items: -    - "{{ cert_path }}" -    - "{{ cert_parent_path }}/ca" - -  # TODO: notify restart openshift-node and/or restart openshift-sdn-node, -  # possibly test service started time against certificate/config file -  # timestamps in openshift-node or openshift-sdn-node to trigger notify -  - name: Sync certs to nodes -    synchronize: -      checksum: yes -      src: "{{ item.src }}" -      dest: "{{ item.dest }}" -      owner: no -      group: no -    with_items: -    - src: "{{ sync_tmpdir }}/{{ cert_rel_path }}" -      dest: "{{ cert_parent_path }}" -    - src: "{{ sync_tmpdir }}/{{ cert_parent_rel_path }}/ca/cert.crt" -      dest: "{{ cert_parent_path }}/ca/cert.crt" -  - local_action: file name={{ sync_tmpdir }} state=absent -    run_once: true -  roles: -    - openshift_node -    #- openshift_sdn_node -    - os_env_extras -    - os_env_extras_node diff --git a/playbooks/aws/openshift-node/launch.yml b/playbooks/aws/openshift-node/launch.yml index b7ef593e7..36aee14ff 100644 --- a/playbooks/aws/openshift-node/launch.yml +++ b/playbooks/aws/openshift-node/launch.yml @@ -4,14 +4,12 @@    connection: local    gather_facts: no +# TODO: modify atomic_ami based on deployment_type    vars:      inst_region: us-east-1      atomic_ami: ami-86781fee      user_data_file: user_data.txt -  vars_files: -    - vars.yml -    tasks:      - name: Launch instances        ec2: @@ -33,7 +31,7 @@        with_items: ec2.instances      - name: Add Name and environment tags to instances -      ec2_tag: "resource={{ item.1.id }} region={{ inst_region }} state=present" +      ec2_tag: resource={{ item.1.id }} region={{ inst_region }} state=present        with_together:          - oo_new_inst_names          - ec2.instances @@ -42,7 +40,7 @@            Name: "{{ item.0 }}"      - name: Add other tags to instances -      ec2_tag: "resource={{ item.id }} region={{ inst_region }} state=present" +      ec2_tag: resource={{ item.id }} region={{ inst_region }} state=present        with_items: ec2.instances        args:          tags: "{{ oo_new_inst_tags }}" @@ -59,7 +57,7 @@          - ec2.instances      - name: Wait for ssh -      wait_for: "port=22 host={{ item.dns_name }}" +      wait_for: port=22 host={{ item.dns_name }}        with_items: ec2.instances      - name: Wait for root user setup diff --git a/playbooks/aws/openshift-node/terminate.yml b/playbooks/aws/openshift-node/terminate.yml index 1c0c77eb7..40ae56f99 100644 --- a/playbooks/aws/openshift-node/terminate.yml +++ b/playbooks/aws/openshift-node/terminate.yml @@ -1,15 +1,15 @@  --- -- name: Populate oo_nodes_to_terminate host group if needed +- name: Populate oo_nodes_to_terminate host group    hosts: localhost    gather_facts: no    tasks: -    - name: Evaluate oo_host_group_exp if it's set -      add_host: "name={{ item }} groups=oo_nodes_to_terminate" -      with_items: "{{ oo_host_group_exp | default('') }}" -      when: oo_host_group_exp is defined +    - name: Evaluate oo_nodes_to_terminate +      add_host: name={{ item }} groups=oo_nodes_to_terminate +      with_items: oo_host_group_exp | default([]) -- name: Gather facts for instances to terminate +- name: Gather dynamic inventory variables for hosts to terminate    hosts: oo_nodes_to_terminate +  gather_facts: no  - name: Terminate instances    hosts: localhost @@ -27,11 +27,12 @@        ignore_errors: yes        register: ec2_term        with_items: host_vars +      when: "'oo_nodes_to_terminate' in groups"      # Fail if any of the instances failed to terminate with an error other      # than 403 Forbidden      - fail: msg=Terminating instance {{ item.item.ec2_id }} failed with message {{ item.msg }} -      when: "item.failed and not item.msg | search(\"error: EC2ResponseError: 403 Forbidden\")" +      when: "'oo_nodes_to_terminate' in groups and item.failed and not item.msg | search(\"error: EC2ResponseError: 403 Forbidden\")"        with_items: ec2_term.results      - name: Stop instance if termination failed @@ -42,6 +43,7 @@        register: ec2_stop        when: item.failed        with_items: ec2_term.results +      when: "'oo_nodes_to_terminate' in groups"      - name: Rename stopped instances        ec2_tag: resource={{ item.item.item.ec2_id }} region={{ item.item.item.ec2_region }} state=present @@ -49,4 +51,5 @@          tags:            Name: "{{ item.item.item.ec2_tag_Name }}-terminate"        with_items: ec2_stop.results +      when: "'oo_nodes_to_terminate' in groups" diff --git a/playbooks/aws/openshift-node/vars.yml b/playbooks/aws/openshift-node/vars.yml deleted file mode 100644 index c196b2fca..000000000 --- a/playbooks/aws/openshift-node/vars.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -openshift_debug_level: 4 -openshift_cluster_id: "{{ cluster_id }}" diff --git a/playbooks/byo/openshift-master/config.yml b/playbooks/byo/openshift-master/config.yml index 706f9285c..f61d277c6 100644 --- a/playbooks/byo/openshift-master/config.yml +++ b/playbooks/byo/openshift-master/config.yml @@ -1,9 +1,15 @@  --- -- name: Gather facts for node hosts -  hosts: nodes +- name: Populate oo_masters_to_config host group +  hosts: localhost +  gather_facts: no +  tasks: +  - add_host: +      name: "{{ item }}" +      groups: oo_masters_to_config +    with_items: groups['masters'] -- name: Configure master instances -  hosts: masters -  roles: -  - openshift_master -  - openshift_sdn_master +- include: ../../common/openshift-master/config.yml +  vars: +    openshift_cluster_id: "{{ cluster_id | default('default') }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}" diff --git a/playbooks/byo/openshift-node/config.yml b/playbooks/byo/openshift-node/config.yml index 69ad7a840..d569827b4 100644 --- a/playbooks/byo/openshift-node/config.yml +++ b/playbooks/byo/openshift-node/config.yml @@ -1,79 +1,21 @@  --- -- name: Gather facts for node hosts -  hosts: nodes -  roles: -  - openshift_facts +- name: Populate oo_nodes_to_config and oo_first_master host groups +  hosts: localhost +  gather_facts: no    tasks: -  # Since the master is registering the nodes before they are configured, we -  # need to make sure to set the node properties beforehand if we do not want -  # the defaults -  - openshift_facts: -      role: 'node' -      local_facts: -        hostname: "{{ openshift_hostname | default(None) }}" -        external_id: "{{ openshift_node_external_id | default(None) }}" -        resources_cpu: "{{ openshfit_node_resources_cpu | default(None) }}" -        resources_memory: "{{ openshfit_node_resources_memory | default(None) }}" -        pod_cidr: "{{ openshfit_node_pod_cidr | default(None) }}" -        labels: "{{ openshfit_node_labels | default(None) }}" -        annotations: "{{ openshfit_node_annotations | default(None) }}" +  - name: Evaluate oo_nodes_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_config +    with_items: groups.nodes +  - name: Evaluate oo_first_master +    add_host: +      name: "{{ groups.masters[0] }}" +      groups: oo_first_master -- name: Register nodes -  hosts: masters[0] +- include: ../../common/openshift-node/config.yml    vars: -    openshift_nodes: "{{ hostvars | oo_select_keys(groups['nodes']) }}" -  roles: -  - openshift_register_nodes -  tasks: -  - name: Create local temp directory for syncing certs -    local_action: command /usr/bin/mktemp -d /tmp/openshift-ansible-XXXXXXX -    register: mktemp - -  - name: Sync master certs to localhost -    synchronize: -      mode: pull -      checksum: yes -      src: /var/lib/openshift/openshift.local.certificates -      dest: "{{ mktemp.stdout }}" - - -- name: Configure node instances -  hosts: nodes -  vars: -    sync_tmpdir: "{{ hostvars[groups['masters'][0]].mktemp.stdout }}" -    cert_parent_rel_path: openshift.local.certificates -    cert_rel_path: "{{ cert_parent_rel_path }}/node-{{ openshift.common.hostname }}" -    cert_base_path: /var/lib/openshift -    cert_parent_path: "{{ cert_base_path }}/{{ cert_parent_rel_path }}" -    cert_path: "{{ cert_base_path }}/{{ cert_rel_path }}" -    openshift_sdn_master_url: http://{{ hostvars[groups['masters'][0]].openshift.common.hostname }}:4001 -  pre_tasks: -  - name: Ensure certificate directories exists -    file: -      path: "{{ item }}" -      state: directory -    with_items: -    - "{{ cert_path }}" -    - "{{ cert_parent_path }}/ca" - -  # TODO: notify restart openshift-node and/or restart openshift-sdn-node, -  # possibly test service started time against certificate/config file -  # timestamps in openshift-node or openshift-sdn-node to trigger notify -  - name: Sync certs to nodes -    synchronize: -      checksum: yes -      src: "{{ item.src }}" -      dest: "{{ item.dest }}" -      owner: no -      group: no -    with_items: -    - src: "{{ sync_tmpdir }}/{{ cert_rel_path }}" -      dest: "{{ cert_parent_path }}" -    - src: "{{ sync_tmpdir }}/{{ cert_parent_rel_path }}/ca/cert.crt" -      dest: "{{ cert_parent_path }}/ca/cert.crt" -  - local_action: file name={{ sync_tmpdir }} state=absent -    run_once: true -  roles: -  - openshift_node -  - openshift_sdn_node +    openshift_cluster_id: "{{ cluster_id | default('default') }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}" diff --git a/playbooks/byo/openshift_facts.yml b/playbooks/byo/openshift_facts.yml new file mode 100644 index 000000000..cd282270f --- /dev/null +++ b/playbooks/byo/openshift_facts.yml @@ -0,0 +1,10 @@ +--- +- name: Gather OpenShift facts +  hosts: all +  gather_facts: no +  roles: +  - openshift_facts +  tasks: +  - openshift_facts: +    register: result +  - debug: var=result diff --git a/playbooks/common/openshift-cluster/config.yml b/playbooks/common/openshift-cluster/config.yml new file mode 100644 index 000000000..14ffa928f --- /dev/null +++ b/playbooks/common/openshift-cluster/config.yml @@ -0,0 +1,4 @@ +--- +- include: ../openshift-master/config.yml + +- include: ../openshift-node/config.yml diff --git a/playbooks/libvirt/openshift-master/filter_plugins b/playbooks/common/openshift-cluster/filter_plugins index 99a95e4ca..99a95e4ca 120000 --- a/playbooks/libvirt/openshift-master/filter_plugins +++ b/playbooks/common/openshift-cluster/filter_plugins diff --git a/playbooks/libvirt/openshift-master/roles b/playbooks/common/openshift-cluster/roles index 20c4c58cf..20c4c58cf 120000 --- a/playbooks/libvirt/openshift-master/roles +++ b/playbooks/common/openshift-cluster/roles diff --git a/playbooks/common/openshift-cluster/set_master_launch_facts_tasks.yml b/playbooks/common/openshift-cluster/set_master_launch_facts_tasks.yml new file mode 100644 index 000000000..118727273 --- /dev/null +++ b/playbooks/common/openshift-cluster/set_master_launch_facts_tasks.yml @@ -0,0 +1,11 @@ +--- +- set_fact: k8s_type="master" + +- name: Generate master instance names(s) +  set_fact: +    scratch_name: "{{ cluster_id }}-{{ k8s_type }}-{{ '%05x' | format(1048576 | random) }}" +  register: master_names_output +  with_sequence: start=1 end={{ num_masters }} + +- set_fact: +    master_names: "{{ master_names_output.results | oo_collect('ansible_facts') | oo_collect('scratch_name') }}" diff --git a/playbooks/common/openshift-cluster/set_node_launch_facts_tasks.yml b/playbooks/common/openshift-cluster/set_node_launch_facts_tasks.yml new file mode 100644 index 000000000..162315d46 --- /dev/null +++ b/playbooks/common/openshift-cluster/set_node_launch_facts_tasks.yml @@ -0,0 +1,11 @@ +--- +- set_fact: k8s_type="node" + +- name: Generate node instance names(s) +  set_fact: +    scratch_name: "{{ cluster_id }}-{{ k8s_type }}-{{ '%05x' | format(1048576 | random) }}" +  register: node_names_output +  with_sequence: start=1 end={{ num_nodes }} + +- set_fact: +    node_names: "{{ node_names_output.results | oo_collect('ansible_facts') | oo_collect('scratch_name') }}" diff --git a/playbooks/common/openshift-cluster/update_repos_and_packages.yml b/playbooks/common/openshift-cluster/update_repos_and_packages.yml new file mode 100644 index 000000000..e92c6f1ee --- /dev/null +++ b/playbooks/common/openshift-cluster/update_repos_and_packages.yml @@ -0,0 +1,7 @@ +--- +- hosts: oo_hosts_to_update +  vars: +    openshift_deployment_type: "{{ deployment_type }}" +  roles: +  - openshift_repos +  - os_update_latest diff --git a/playbooks/common/openshift-master/config.yml b/playbooks/common/openshift-master/config.yml new file mode 100644 index 000000000..05822d118 --- /dev/null +++ b/playbooks/common/openshift-master/config.yml @@ -0,0 +1,19 @@ +--- +- name: Configure master instances +  hosts: oo_masters_to_config +  vars: +    openshift_sdn_master_url: https://{{ openshift.common.hostname }}:4001 +  roles: +  - openshift_master +  - { role: openshift_sdn_master, when: openshift.common.use_openshift_sdn | bool } +  tasks: +  - name: Create group for deployment type +    group_by: key=oo_masters_deployment_type_{{ openshift.common.deployment_type }} +    changed_when: False + +# Additional instance config for online deployments +- name: Additional instance config +  hosts: oo_masters_deployment_type_online +  roles: +  - pods +  - os_env_extras diff --git a/playbooks/libvirt/openshift-node/filter_plugins b/playbooks/common/openshift-master/filter_plugins index 99a95e4ca..99a95e4ca 120000 --- a/playbooks/libvirt/openshift-node/filter_plugins +++ b/playbooks/common/openshift-master/filter_plugins diff --git a/playbooks/common/openshift-master/roles b/playbooks/common/openshift-master/roles new file mode 120000 index 000000000..e2b799b9d --- /dev/null +++ b/playbooks/common/openshift-master/roles @@ -0,0 +1 @@ +../../../roles/
\ No newline at end of file diff --git a/playbooks/common/openshift-node/config.yml b/playbooks/common/openshift-node/config.yml new file mode 100644 index 000000000..c82d69c28 --- /dev/null +++ b/playbooks/common/openshift-node/config.yml @@ -0,0 +1,121 @@ +--- +- name: Gather and set facts for node hosts +  hosts: oo_nodes_to_config +  roles: +  - openshift_facts +  tasks: +  # Since the master is registering the nodes before they are configured, we +  # need to make sure to set the node properties beforehand if we do not want +  # the defaults +  - openshift_facts: +      role: "{{ item.role }}" +      local_facts: "{{ item.local_facts }}" +    with_items: +      - role: common +        local_facts: +          hostname: "{{ openshift_hostname | default(None) }}" +          public_hostname: "{{ openshift_public_hostname | default(None) }}" +      - role: node +        local_facts: +          external_id: "{{ openshift_node_external_id | default(None) }}" +          resources_cpu: "{{ openshift_node_resources_cpu | default(None) }}" +          resources_memory: "{{ openshift_node_resources_memory | default(None) }}" +          pod_cidr: "{{ openshift_node_pod_cidr | default(None) }}" +          labels: "{{ openshift_node_labels | default(None) }}" +          annotations: "{{ openshift_node_annotations | default(None) }}" +          deployment_type: "{{ openshift_deployment_type }}" + + +- name: Create temp directory for syncing certs +  hosts: localhost +  gather_facts: no +  tasks: +  - name: Create local temp directory for syncing certs +    local_action: command mktemp -d /tmp/openshift-ansible-XXXXXXX +    register: mktemp +    changed_when: False + + +- name: Register nodes +  hosts: oo_first_master +  vars: +    openshift_nodes: "{{ hostvars | oo_select_keys(groups['oo_nodes_to_config']) }}" +    sync_tmpdir: "{{ hostvars.localhost.mktemp.stdout }}" +  roles: +  - openshift_register_nodes +  tasks: +  - name: Create the temp directory on the master +    file: +      path: "{{ sync_tmpdir }}" +      owner: "{{ ansible_ssh_user }}" +      mode: 0700 +      state: directory +    changed_when: False + +  - name: Create a tarball of the node config directories +    command: tar -czvf {{ sync_tmpdir }}/{{ item.openshift.common.hostname }}.tgz ./ +    args: +      chdir: "{{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}" +    with_items: openshift_nodes +    changed_when: False + +  - name: Retrieve the node config tarballs from the master +    fetch: +      src: "{{ sync_tmpdir }}/{{ item.openshift.common.hostname }}.tgz" +      dest: "{{ sync_tmpdir }}/" +      flat: yes +      fail_on_missing: yes +      validate_checksum: yes +    with_items: openshift_nodes +    changed_when: False + +  - name: Remove the temp directory on the master +    file: +      path: "{{ sync_tmpdir }}" +      state: absent +    changed_when: False + + +- name: Configure node instances +  hosts: oo_nodes_to_config +  gather_facts: no +  vars: +    sync_tmpdir: "{{ hostvars.localhost.mktemp.stdout }}" +    openshift_sdn_master_url: "https://{{ hostvars[groups['oo_first_master'][0]].openshift.common.hostname }}:4001" +  pre_tasks: +  - name: Ensure certificate directory exists +    file: +      path: "{{ openshift_node_cert_dir }}" +      state: directory + +  # TODO: notify restart openshift-node and/or restart openshift-sdn-node, +  # possibly test service started time against certificate/config file +  # timestamps in openshift-node or openshift-sdn-node to trigger notify +  - name: Unarchive the tarball on the node +    unarchive: +      src: "{{ sync_tmpdir }}/{{ openshift.common.hostname }}.tgz" +      dest: "{{ openshift_node_cert_dir }}" +  roles: +  - openshift_node +  - { role: openshift_sdn_node, when: openshift.common.use_openshift_sdn | bool } +  tasks: +  - name: Create group for deployment type +    group_by: key=oo_nodes_deployment_type_{{ openshift.common.deployment_type }} +    changed_when: False + + +- name: Delete temporary directory +  hosts: localhost +  gather_facts: no +  tasks: +  - file: name={{ mktemp.stdout }} state=absent +    changed_when: False + + +# Additional config for online type deployments +- name: Additional instance config +  hosts: oo_nodes_deployment_type_online +  gather_facts: no +  roles: +  - os_env_extras +  - os_env_extras_node diff --git a/playbooks/common/openshift-node/filter_plugins b/playbooks/common/openshift-node/filter_plugins new file mode 120000 index 000000000..99a95e4ca --- /dev/null +++ b/playbooks/common/openshift-node/filter_plugins @@ -0,0 +1 @@ +../../../filter_plugins
\ No newline at end of file diff --git a/playbooks/common/openshift-node/roles b/playbooks/common/openshift-node/roles new file mode 120000 index 000000000..e2b799b9d --- /dev/null +++ b/playbooks/common/openshift-node/roles @@ -0,0 +1 @@ +../../../roles/
\ No newline at end of file diff --git a/playbooks/gce/openshift-cluster/config.yml b/playbooks/gce/openshift-cluster/config.yml new file mode 100644 index 000000000..8b8490246 --- /dev/null +++ b/playbooks/gce/openshift-cluster/config.yml @@ -0,0 +1,37 @@ +--- +# TODO: fix firewall related bug with GCE and origin, since GCE is overriding +# /etc/sysconfig/iptables +- name: Populate oo_masters_to_config host group +  hosts: localhost +  gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - name: Evaluate oo_masters_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_masters_to_config +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type-{{ cluster_id }}-openshift-master"] | default([]) +  - name: Evaluate oo_nodes_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_config +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type-{{ cluster_id }}-openshift-node"] | default([]) +  - name: Evaluate oo_first_master +    add_host: +      name: "{{ groups['tag_env-host-type-' ~ cluster_id ~ '-openshift-master'][0] }}" +      groups: oo_first_master +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    when: "'tag_env-host-type-{{ cluster_id }}-openshift-master' in groups" + +- include: ../../common/openshift-cluster/config.yml +  vars: +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}" +    openshift_hostname: "{{ gce_private_ip }}" diff --git a/playbooks/gce/openshift-cluster/launch.yml b/playbooks/gce/openshift-cluster/launch.yml index 14cdd2537..34a5a0b94 100644 --- a/playbooks/gce/openshift-cluster/launch.yml +++ b/playbooks/gce/openshift-cluster/launch.yml @@ -4,59 +4,25 @@    connection: local    gather_facts: no    vars_files: -      - vars.yml +  - vars.yml    tasks: -    - set_fact: k8s_type="master" - -    - name: Generate master instance names(s) -      set_fact: scratch={{ cluster_id }}-{{ k8s_type }}-{{ '%05x' |format( 1048576 |random) }} -      register: master_names_output -      with_sequence: start=1 end={{ num_masters }} - -    # These set_fact's cannot be combined -    - set_fact: -        master_names_string: "{% for item in master_names_output.results %}{{ item.ansible_facts.scratch }} {% endfor %}" - -    - set_fact: -        master_names: "{{ master_names_string.strip().split(' ') }}" - -    - include: launch_instances.yml -      vars: -        instances: "{{ master_names }}" -        cluster: "{{ cluster_id }}" -        type: "{{ k8s_type }}" - -    - set_fact: k8s_type="node" - -    - name: Generate node instance names(s) -      set_fact: scratch={{ cluster_id }}-{{ k8s_type }}-{{ '%05x' |format( 1048576 |random) }} -      register: node_names_output -      with_sequence: start=1 end={{ num_nodes }} - -    # These set_fact's cannot be combined -    - set_fact: -        node_names_string: "{% for item in node_names_output.results %}{{ item.ansible_facts.scratch }} {% endfor %}" - -    - set_fact: -        node_names: "{{ node_names_string.strip().split(' ') }}" - -    - include: launch_instances.yml -      vars: -        instances: "{{ node_names }}" -        cluster: "{{ cluster_id }}" -        type: "{{ k8s_type }}" - -- hosts: "tag_env-{{ cluster_id }}" -  roles: -  - openshift_repos -  - os_update_latest - -- include: ../openshift-master/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type-{{ cluster_id }}-openshift-master\"]" - -- include: ../openshift-node/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type-{{ cluster_id }}-openshift-node\"]" +  - fail: msg="Deployment type not supported for libvirt provider yet" +    when: deployment_type == 'enterprise' + +  - include: ../../common/openshift-cluster/set_master_launch_facts_tasks.yml +  - include: tasks/launch_instances.yml +    vars: +      instances: "{{ master_names }}" +      cluster: "{{ cluster_id }}" +      type: "{{ k8s_type }}" + +  - include: ../../common/openshift-cluster/set_node_launch_facts_tasks.yml +  - include: tasks/launch_instances.yml +    vars: +      instances: "{{ node_names }}" +      cluster: "{{ cluster_id }}" +      type: "{{ k8s_type }}" + +- include: update.yml  - include: list.yml diff --git a/playbooks/gce/openshift-cluster/list.yml b/playbooks/gce/openshift-cluster/list.yml index 1124b0ea3..bab2fb9f8 100644 --- a/playbooks/gce/openshift-cluster/list.yml +++ b/playbooks/gce/openshift-cluster/list.yml @@ -2,16 +2,23 @@  - name: Generate oo_list_hosts group    hosts: localhost    gather_facts: no +  vars_files: +  - vars.yml    tasks:    - set_fact: scratch_group=tag_env-{{ cluster_id }}      when: cluster_id != ''    - set_fact: scratch_group=all -    when: scratch_group is not defined -  - add_host: name={{ item }} groups=oo_list_hosts -    with_items: groups[scratch_group] | difference(['localhost']) | difference(groups.status_terminated) +    when: cluster_id == '' +  - add_host: +      name: "{{ item }}" +      groups: oo_list_hosts +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost']) | difference(groups.status_terminated)  - name: List Hosts    hosts: oo_list_hosts    gather_facts: no    tasks: -  - debug: msg="public:{{hostvars[inventory_hostname].gce_public_ip}} private:{{hostvars[inventory_hostname].gce_private_ip}}" +  - debug: +      msg: "public ip:{{ hostvars[inventory_hostname].gce_public_ip }} private ip:{{ hostvars[inventory_hostname].gce_private_ip }} deployment-type: {{ hostvars[inventory_hostname].group_names | oo_get_deployment_type_from_groups }}" diff --git a/playbooks/gce/openshift-cluster/launch_instances.yml b/playbooks/gce/openshift-cluster/tasks/launch_instances.yml index b4f33bd87..a68edefae 100644 --- a/playbooks/gce/openshift-cluster/launch_instances.yml +++ b/playbooks/gce/openshift-cluster/tasks/launch_instances.yml @@ -2,41 +2,39 @@  # TODO: when we are ready to go to ansible 1.9+ support only, we can update to  # the gce task to use the disk_auto_delete parameter to avoid having to delete  # the disk as a separate step on termination - -- set_fact: -    machine_type: "{{ lookup('env', 'gce_machine_type') |default('n1-standard-1', true) }}" -    machine_image: "{{ lookup('env', 'gce_machine_image') |default('libra-rhel7', true) }}" -  - name: Launch instance(s)    gce:      instance_names: "{{ instances }}" -    machine_type: "{{ machine_type }}" -    image: "{{ machine_image }}" +    machine_type: "{{ lookup('env', 'gce_machine_type') | default('n1-standard-1', true) }}" +    image: "{{ lookup('env', 'gce_machine_image') | default(deployment_vars[deployment_type].image, true) }}"      service_account_email: "{{ lookup('env', 'gce_service_account_email_address') }}"      pem_file: "{{ lookup('env', 'gce_service_account_pem_file_path') }}"      project_id: "{{ lookup('env', 'gce_project_id') }}"      tags: -      - "created-by-{{ lookup('env', 'LOGNAME') |default(cluster, true) }}" -      - "env-{{ cluster }}" -      - "host-type-{{ type }}" -      - "env-host-type-{{ cluster }}-openshift-{{ type }}" +      - created-by-{{ lookup('env', 'LOGNAME') |default(cluster, true) }} +      - env-{{ cluster }} +      - host-type-{{ type }} +      - env-host-type-{{ cluster }}-openshift-{{ type }} +      - deployment-type-{{ deployment_type }}    register: gce  - name: Add new instances to groups and set variables needed    add_host:      hostname: "{{ item.name }}"      ansible_ssh_host: "{{ item.public_ip }}" +    ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +    ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}"      groups: "{{ item.tags | oo_prepend_strings_in_list('tag_') | join(',') }}"      gce_public_ip: "{{ item.public_ip }}"      gce_private_ip: "{{ item.private_ip }}"    with_items: gce.instance_data  - name: Wait for ssh -  wait_for: "port=22 host={{ item.public_ip }}" +  wait_for: port=22 host={{ item.public_ip }}    with_items: gce.instance_data -- name: Wait for root user setup -  command: "ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null root@{{ item.public_ip }} echo root user is setup" +- name: Wait for user setup +  command: "ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null {{ hostvars[item.name].ansible_ssh_user }}@{{ item.public_ip }} echo {{ hostvars[item.name].ansible_ssh_user }} user is setup"    register: result    until: result.rc == 0    retries: 20 diff --git a/playbooks/gce/openshift-cluster/terminate.yml b/playbooks/gce/openshift-cluster/terminate.yml index 0281ae953..abe6a4c95 100644 --- a/playbooks/gce/openshift-cluster/terminate.yml +++ b/playbooks/gce/openshift-cluster/terminate.yml @@ -1,20 +1,34 @@  ---  - name: Terminate instance(s)    hosts: localhost - +  gather_facts: no    vars_files: -    - vars.yml +  - vars.yml +  tasks: +  - set_fact: scratch_group=tag_env-host-type-{{ cluster_id }}-openshift-node +  - add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_terminate +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost']) | difference(groups.status_terminated) + +  - set_fact: scratch_group=tag_env-host-type-{{ cluster_id }}-openshift-master +  - add_host: +      name: "{{ item }}" +      groups: oo_masters_to_terminate +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost']) | difference(groups.status_terminated)  - include: ../openshift-node/terminate.yml    vars: -    oo_host_group_exp: 'groups["tag_env-host-type-{{ cluster_id }}-openshift-node"]'      gce_service_account_email: "{{ lookup('env', 'gce_service_account_email_address') }}"      gce_pem_file: "{{ lookup('env', 'gce_service_account_pem_file_path') }}"      gce_project_id: "{{ lookup('env', 'gce_project_id') }}"  - include: ../openshift-master/terminate.yml    vars: -    oo_host_group_exp: 'groups["tag_env-host-type-{{ cluster_id }}-openshift-master"]'      gce_service_account_email: "{{ lookup('env', 'gce_service_account_email_address') }}"      gce_pem_file: "{{ lookup('env', 'gce_service_account_pem_file_path') }}"      gce_project_id: "{{ lookup('env', 'gce_project_id') }}" diff --git a/playbooks/gce/openshift-cluster/update.yml b/playbooks/gce/openshift-cluster/update.yml index 973e4c3ef..9ebf39a13 100644 --- a/playbooks/gce/openshift-cluster/update.yml +++ b/playbooks/gce/openshift-cluster/update.yml @@ -1,13 +1,18 @@  --- -- hosts: "tag_env-{{ cluster_id }}" -  roles: -  - openshift_repos -  - os_update_latest +- name: Populate oo_hosts_to_update group +  hosts: localhost +  gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - name: Evaluate oo_hosts_to_update +    add_host: +      name: "{{ item }}" +      groups: oo_hosts_to_update +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user | default(ansible_ssh_user, true) }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type-{{ cluster_id }}-openshift-master"] | union(groups["tag_env-host-type-{{ cluster_id }}-openshift-node"]) | default([]) -- include: ../openshift-master/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type-{{ cluster_id }}-openshift-master\"]" +- include: ../../common/openshift-cluster/update_repos_and_packages.yml -- include: ../openshift-node/config.yml -  vars: -    oo_host_group_exp: "groups[\"tag_env-host-type-{{ cluster_id }}-openshift-node\"]" +- include: config.yml diff --git a/playbooks/gce/openshift-cluster/vars.yml b/playbooks/gce/openshift-cluster/vars.yml index ed97d539c..ae33083b9 100644 --- a/playbooks/gce/openshift-cluster/vars.yml +++ b/playbooks/gce/openshift-cluster/vars.yml @@ -1 +1,15 @@  --- +deployment_vars: +  origin: +    image: centos-7 +    ssh_user: +    sudo: yes +  online: +    image: libra-rhel7 +    ssh_user: root +    sudo: no +  enterprise: +    image: rhel-7 +    ssh_user: +    sudo: yes + diff --git a/playbooks/gce/openshift-master/config.yml b/playbooks/gce/openshift-master/config.yml index 857da0763..af6000bc8 100644 --- a/playbooks/gce/openshift-master/config.yml +++ b/playbooks/gce/openshift-master/config.yml @@ -1,20 +1,18 @@  --- -- name: master/config.yml, populate oo_masters_to_config host group if needed +- name: Populate oo_masters_to_config host group    hosts: localhost    gather_facts: no    tasks: -  - name: "Evaluate oo_host_group_exp if it's set" -    add_host: "name={{ item }} groups=oo_masters_to_config" -    with_items: "{{ oo_host_group_exp | default('') }}" -    when: oo_host_group_exp is defined +  - name: Evaluate oo_masters_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_masters_to_config +      ansible_ssh_user: root +    with_items: oo_host_group_exp | default([]) -- name: "Configure instances" -  hosts: oo_masters_to_config +- include: ../../common/openshift-master/config.yml    vars: +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}"      openshift_hostname: "{{ gce_private_ip }}" -  vars_files: -  - vars.yml -  roles: -    - openshift_master -    - pods -    - os_env_extras diff --git a/playbooks/gce/openshift-master/launch.yml b/playbooks/gce/openshift-master/launch.yml index 287596002..ef10b6cf0 100644 --- a/playbooks/gce/openshift-master/launch.yml +++ b/playbooks/gce/openshift-master/launch.yml @@ -8,14 +8,12 @@    connection: local    gather_facts: no +# TODO: modify image based on deployment_type    vars:      inst_names: "{{ oo_new_inst_names }}"      machine_type: n1-standard-1      image: libra-rhel7 -  vars_files: -      - vars.yml -    tasks:      - name: Launch instances        gce: @@ -37,7 +35,7 @@        with_items: gce.instance_data      - name: Wait for ssh -      wait_for: "port=22 host={{ item.public_ip }}" +      wait_for: port=22 host={{ item.public_ip }}        with_items: gce.instance_data      - name: Wait for root user setup diff --git a/playbooks/gce/openshift-master/terminate.yml b/playbooks/gce/openshift-master/terminate.yml index 8319774f8..452ac5199 100644 --- a/playbooks/gce/openshift-master/terminate.yml +++ b/playbooks/gce/openshift-master/terminate.yml @@ -3,10 +3,9 @@    hosts: localhost    gather_facts: no    tasks: -    - name: Evaluate oo_host_group_exp if it's set -      add_host: "name={{ item }} groups=oo_masters_to_terminate" -      with_items: "{{ oo_host_group_exp | default('') }}" -      when: oo_host_group_exp is defined +    - name: Evaluate oo_masters_to_terminate +      add_host: name={{ item }} groups=oo_masters_to_terminate +      with_items: oo_host_group_exp | default([])  - name: Terminate master instances    hosts: localhost @@ -22,6 +21,7 @@          instance_names: "{{ groups['oo_masters_to_terminate'] }}"          disks: "{{ groups['oo_masters_to_terminate'] }}"        register: gce +      when: "'oo_masters_to_terminate' in groups"      - name: Remove disks of instances        gce_pd: @@ -32,5 +32,4 @@          zone: "{{ gce.zone }}"          state: absent        with_items: gce.instance_names - - +      when: "'oo_masters_to_terminate' in groups" diff --git a/playbooks/gce/openshift-master/vars.yml b/playbooks/gce/openshift-master/vars.yml deleted file mode 100644 index c196b2fca..000000000 --- a/playbooks/gce/openshift-master/vars.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -openshift_debug_level: 4 -openshift_cluster_id: "{{ cluster_id }}" diff --git a/playbooks/gce/openshift-node/config.yml b/playbooks/gce/openshift-node/config.yml index 771cc3a94..5b1601176 100644 --- a/playbooks/gce/openshift-node/config.yml +++ b/playbooks/gce/openshift-node/config.yml @@ -1,100 +1,24 @@  --- -- name: node/config.yml, populate oo_nodes_to_config host group if needed +- name: Populate oo_nodes_to_config and oo_first_master host groups    hosts: localhost    gather_facts: no    tasks: -  - name: Evaluate oo_host_group_exp -    add_host: "name={{ item }} groups=oo_nodes_to_config" -    with_items: "{{ oo_host_group_exp | default('') }}" -    when: oo_host_group_exp is defined -  - add_host: +  - name: Evaluate oo_nodes_to_config +    add_host: +      name: "{{ item }}" +      groups: oo_nodes_to_config +      ansible_ssh_user: root +    with_items: oo_host_group_exp | default([]) +  - name: Evaluate oo_first_master +    add_host:        name: "{{ groups['tag_env-host-type-' ~ cluster_id ~ '-openshift-master'][0] }}"        groups: oo_first_master -    when: oo_host_group_exp is defined +      ansible_ssh_user: root -- name: Gather and set facts for hosts to configure -  hosts: oo_nodes_to_config -  roles: -  - openshift_facts -  tasks: -  # Since the master is registering the nodes before they are configured, we -  # need to make sure to set the node properties beforehand if we do not want -  # the defaults -  - openshift_facts: -      role: "{{ item.role }}" -      local_facts: "{{ item.local_facts }}" -    with_items: -    - role: common -      local_facts: -        hostname: "{{ gce_private_ip }}" -    - role: node -      local_facts: -        external_id: "{{ openshift_node_external_id | default(None) }}" -        resources_cpu: "{{ openshfit_node_resources_cpu | default(None) }}" -        resources_memory: "{{ openshfit_node_resources_memory | default(None) }}" -        pod_cidr: "{{ openshfit_node_pod_cidr | default(None) }}" -        labels: "{{ openshfit_node_labels | default(None) }}" -        annotations: "{{ openshfit_node_annotations | default(None) }}" - - -- name: Register nodes -  hosts: oo_first_master -  vars: -    openshift_nodes: "{{ hostvars -          | oo_select_keys(groups['oo_nodes_to_config']) }}" -  roles: -  - openshift_register_nodes -  tasks: -  - name: Create local temp directory for syncing certs -    local_action: command /usr/bin/mktemp -d /tmp/openshift-ansible-XXXXXXX -    register: mktemp - -  - name: Sync master certs to localhost -    synchronize: -      mode: pull -      checksum: yes -      src: /var/lib/openshift/openshift.local.certificates -      dest: "{{ mktemp.stdout }}" - -- name: Configure instances -  hosts: oo_nodes_to_config -  vars_files: -  - vars.yml +- include: ../../common/openshift-node/config.yml    vars: -    sync_tmpdir: "{{ hostvars[groups['oo_first_master'][0]].mktemp.stdout }}" -    cert_parent_rel_path: openshift.local.certificates -    cert_rel_path: "{{ cert_parent_rel_path }}/node-{{ openshift.common.hostname }}" -    cert_base_path: /var/lib/openshift -    cert_parent_path: "{{ cert_base_path }}/{{ cert_parent_rel_path }}" -    cert_path: "{{ cert_base_path }}/{{ cert_rel_path }}" -  pre_tasks: -  - name: Ensure certificate directories exists -    file: -      path: "{{ item }}" -      state: directory -    with_items: -    - "{{ cert_path }}" -    - "{{ cert_parent_path }}/ca" - -  # TODO: notify restart openshift-node and/or restart openshift-sdn-node, -  # possibly test service started time against certificate/config file -  # timestamps in openshift-node or openshift-sdn-node to trigger notify -  - name: Sync certs to nodes -    synchronize: -      checksum: yes -      src: "{{ item.src }}" -      dest: "{{ item.dest }}" -      owner: no -      group: no -    with_items: -    - src: "{{ sync_tmpdir }}/{{ cert_rel_path }}" -      dest: "{{ cert_parent_path }}" -    - src: "{{ sync_tmpdir }}/{{ cert_parent_rel_path }}/ca/cert.crt" -      dest: "{{ cert_parent_path }}/ca/cert.crt" -  - local_action: file name={{ sync_tmpdir }} state=absent -    run_once: true -  roles: -    - openshift_node -    - os_env_extras -    - os_env_extras_node +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}" +    openshift_hostname: "{{ gce_private_ip }}" diff --git a/playbooks/gce/openshift-node/launch.yml b/playbooks/gce/openshift-node/launch.yml index 73d0478ab..086ba58bc 100644 --- a/playbooks/gce/openshift-node/launch.yml +++ b/playbooks/gce/openshift-node/launch.yml @@ -8,14 +8,12 @@    connection: local    gather_facts: no +# TODO: modify image based on deployment_type    vars:      inst_names: "{{ oo_new_inst_names }}"      machine_type: n1-standard-1      image: libra-rhel7 -  vars_files: -      - vars.yml -    tasks:      - name: Launch instances        gce: @@ -37,7 +35,7 @@        with_items: gce.instance_data      - name: Wait for ssh -      wait_for: "port=22 host={{ item.public_ip }}" +      wait_for: port=22 host={{ item.public_ip }}        with_items: gce.instance_data      - name: Wait for root user setup diff --git a/playbooks/gce/openshift-node/terminate.yml b/playbooks/gce/openshift-node/terminate.yml index 7d71dfcab..357e0c295 100644 --- a/playbooks/gce/openshift-node/terminate.yml +++ b/playbooks/gce/openshift-node/terminate.yml @@ -3,10 +3,9 @@    hosts: localhost    gather_facts: no    tasks: -    - name: Evaluate oo_host_group_exp if it's set -      add_host: "name={{ item }} groups=oo_nodes_to_terminate" -      with_items: "{{ oo_host_group_exp | default('') }}" -      when: oo_host_group_exp is defined +    - name: Evaluate oo_nodes_to_terminate +      add_host: name={{ item }} groups=oo_nodes_to_terminate +      with_items: oo_host_group_exp | default([])  - name: Terminate node instances    hosts: localhost @@ -22,6 +21,7 @@          instance_names: "{{ groups['oo_nodes_to_terminate'] }}"          disks: "{{ groups['oo_nodes_to_terminate'] }}"        register: gce +      when: "'oo_nodes_to_terminate' in groups"      - name: Remove disks of instances        gce_pd: @@ -32,5 +32,4 @@          zone: "{{ gce.zone }}"          state: absent        with_items: gce.instance_names - - +      when: "'oo_nodes_to_terminate' in groups" diff --git a/playbooks/gce/openshift-node/vars.yml b/playbooks/gce/openshift-node/vars.yml deleted file mode 100644 index c196b2fca..000000000 --- a/playbooks/gce/openshift-node/vars.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -openshift_debug_level: 4 -openshift_cluster_id: "{{ cluster_id }}" diff --git a/playbooks/libvirt/openshift-cluster/config.yml b/playbooks/libvirt/openshift-cluster/config.yml new file mode 100644 index 000000000..faf278b10 --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/config.yml @@ -0,0 +1,38 @@ +--- +# TODO: need to figure out a plan for setting hostname, currently the default +# is localhost, so no hostname value (or public_hostname) value is getting +# assigned + +- name: Populate oo_masters_to_config host group +  hosts: localhost +  gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - name: Evaluate oo_masters_to_config +    add_host: +      name: "{{ item }}" +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +      groups: oo_masters_to_config +    with_items: groups["tag_env-host-type-{{ cluster_id }}-openshift-master"] | default([]) +  - name: Evaluate oo_nodes_to_config +    add_host: +      name: "{{ item }}" +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +      groups: oo_nodes_to_config +    with_items: groups["tag_env-host-type-{{ cluster_id }}-openshift-node"] | default([]) +  - name: Evaluate oo_first_master +    add_host: +      name: "{{ groups['tag_env-host-type-' ~ cluster_id ~ '-openshift-master'][0] }}" +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +      groups: oo_first_master +    when: "'tag_env-host-type-{{ cluster_id }}-openshift-master' in groups" + +- include: ../../common/openshift-cluster/config.yml +  vars: +    openshift_cluster_id: "{{ cluster_id }}" +    openshift_debug_level: 4 +    openshift_deployment_type: "{{ deployment_type }}" diff --git a/playbooks/libvirt/openshift-cluster/launch.yml b/playbooks/libvirt/openshift-cluster/launch.yml index 6f2df33af..a7ddc1e7e 100644 --- a/playbooks/libvirt/openshift-cluster/launch.yml +++ b/playbooks/libvirt/openshift-cluster/launch.yml @@ -1,65 +1,36 @@ +---  - name: Launch instance(s)    hosts: localhost -  connection: local    gather_facts: no - -  vars: -    libvirt_storage_pool_path: "{{ lookup('env','HOME') }}/libvirt-storage-pool-openshift" -    libvirt_storage_pool: 'openshift' -    libvirt_uri: 'qemu:///system' -    vars_files: -    - vars.yml - +  - vars.yml +  vars: +    os_libvirt_storage_pool: "{{ libvirt_storage_pool | default('images') }}" +    os_libvirt_storage_pool_path: "{{ libvirt_storage_pool_path | default('/var/lib/libvirt/images') }}" +    os_libvirt_network: "{{ libvirt_network | default('default') }}" +    image_url: "{{ deployment_vars[deployment_type].image.url }}" +    image_sha256: "{{ deployment_vars[deployment_type].image.sha256 }}" +    image_name: "{{ deployment_vars[deployment_type].image.name }}"    tasks: -    - set_fact: -        k8s_type: master - -    - name: Generate master instance name(s) -      set_fact: -        scratch_name: "{{ cluster_id }}-{{ k8s_type }}-{{ '%05x' | format( 1048576 | random ) }}" -      register: master_names_output -      with_sequence: start=1 end='{{ num_masters }}' +  - fail: msg="Deployment type not supported for libvirt provider yet" +    when: deployment_type in ['online', 'enterprise'] -    - set_fact: -        master_names: "{{ master_names_output.results | oo_collect('ansible_facts') | oo_collect('scratch_name') }}" +  - include: tasks/configure_libvirt.yml -    - include: launch_instances.yml -      vars: -        instances: '{{ master_names }}' -        cluster: '{{ cluster_id }}' -        type: '{{ k8s_type }}' -        group_name: 'tag_env-host-type-{{ cluster_id }}-openshift-master' +  - include: ../../common/openshift-cluster/set_master_launch_facts_tasks.yml +  - include: tasks/launch_instances.yml +    vars: +      instances: "{{ master_names }}" +      cluster: "{{ cluster_id }}" +      type: "{{ k8s_type }}" -    - set_fact: -        k8s_type: node +  - include: ../../common/openshift-cluster/set_node_launch_facts_tasks.yml +  - include: tasks/launch_instances.yml +    vars: +      instances: "{{ node_names }}" +      cluster: "{{ cluster_id }}" +      type: "{{ k8s_type }}" -    - name: Generate node instance name(s) -      set_fact: -        scratch_name: "{{ cluster_id }}-{{ k8s_type }}-{{ '%05x' | format( 1048576 | random ) }}" -      register: node_names_output -      with_sequence: start=1 end='{{ num_nodes }}' +- include: update.yml -    - set_fact: -        node_names: "{{ node_names_output.results | oo_collect('ansible_facts') | oo_collect('scratch_name') }}" - -    - include: launch_instances.yml -      vars: -        instances: '{{ node_names }}' -        cluster: '{{ cluster_id }}' -        type: '{{ k8s_type }}' - -- hosts: 'tag_env-{{ cluster_id }}' -  roles: -    - openshift_repos -    - os_update_latest - -- include: ../openshift-master/config.yml -  vars: -    oo_host_group_exp: 'groups["tag_env-host-type-{{ cluster_id }}-openshift-master"]' -    oo_env: '{{ cluster_id }}' - -- include: ../openshift-node/config.yml -  vars: -    oo_host_group_exp: 'groups["tag_env-host-type-{{ cluster_id }}-openshift-node"]' -    oo_env: '{{ cluster_id }}' +- include: list.yml diff --git a/playbooks/libvirt/openshift-cluster/launch_instances.yml b/playbooks/libvirt/openshift-cluster/launch_instances.yml deleted file mode 100644 index 3bbcae981..000000000 --- a/playbooks/libvirt/openshift-cluster/launch_instances.yml +++ /dev/null @@ -1,102 +0,0 @@ -- name: Create the libvirt storage directory for openshift -  file: -    dest: '{{ libvirt_storage_pool_path }}' -    state: directory - -- name: Download Base Cloud image -  get_url: -    url: '{{ base_image_url }}' -    sha256sum: '{{ base_image_sha256 }}' -    dest: '{{ libvirt_storage_pool_path }}/{{ base_image_name }}' - -- name: Create the cloud-init config drive path -  file: -    dest: '{{ libvirt_storage_pool_path }}/{{ item }}_configdrive/openstack/latest' -    state: directory -  with_items: '{{ instances }}' - -- name: Create the cloud-init config drive files -  template: -    src: '{{ item[1] }}' -    dest: '{{ libvirt_storage_pool_path }}/{{ item[0] }}_configdrive/openstack/latest/{{ item[1] }}' -  with_nested: -    - '{{ instances }}' -    - [ user-data, meta-data ] - -- name: Create the cloud-init config drive -  command: 'genisoimage -output {{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso -volid cidata -joliet -rock user-data meta-data' -  args: -    chdir: '{{ libvirt_storage_pool_path }}/{{ item }}_configdrive/openstack/latest' -    creates: '{{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso' -  with_items: '{{ instances }}' - -- name: Create the libvirt storage pool for openshift -  command: 'virsh -c {{ libvirt_uri }} pool-create-as {{ libvirt_storage_pool }} dir --target {{ libvirt_storage_pool_path }}' -  ignore_errors: yes - -- name: Refresh the libvirt storage pool for openshift -  command: 'virsh -c {{ libvirt_uri }} pool-refresh {{ libvirt_storage_pool }}' - -- name: Create VMs drives -  command: 'virsh -c {{ libvirt_uri }} vol-create-as {{ libvirt_storage_pool }} {{ item }}.qcow2 10G --format qcow2 --backing-vol {{ base_image_name }} --backing-vol-format qcow2' -  with_items: '{{ instances }}' - -- name: Create VMs -  virt: -    name: '{{ item }}' -    command: define -    xml: "{{ lookup('template', '../templates/domain.xml') }}" -    uri: '{{ libvirt_uri }}' -  with_items: '{{ instances }}' - -- name: Start VMs -  virt: -    name: '{{ item }}' -    state: running -    uri: '{{ libvirt_uri }}' -  with_items: '{{ instances }}' - -- name: Collect MAC addresses of the VMs -  shell: 'virsh -c {{ libvirt_uri }} dumpxml {{ item }} | xmllint --xpath "string(//domain/devices/interface/mac/@address)" -' -  register: scratch_mac -  with_items: '{{ instances }}' - -- name: Wait for the VMs to get an IP -  command: "egrep -c '{{ scratch_mac.results | oo_collect('stdout') | join('|') }}' /proc/net/arp" -  ignore_errors: yes -  register: nb_allocated_ips -  until: nb_allocated_ips.stdout == '{{ instances | length }}' -  retries: 30 -  delay: 1 - -- name: Collect IP addresses of the VMs -  shell: "awk '/{{ item.stdout }}/ {print $1}' /proc/net/arp" -  register: scratch_ip -  with_items: '{{ scratch_mac.results }}' - -- set_fact: -    ips: "{{ scratch_ip.results | oo_collect('stdout') }}" - -- name: Add new instances -  add_host: -    hostname: '{{ item.0 }}' -    ansible_ssh_host: '{{ item.1 }}' -    ansible_ssh_user: root -    groups: 'tag_env-{{ cluster }}, tag_host-type-{{ type }}, tag_env-host-type-{{ cluster }}-openshift-{{ type }}' -  with_together: -    - instances -    - ips - -- name: Wait for ssh -  wait_for: -    host: '{{ item }}' -    port: 22 -  with_items: ips - -- name: Wait for root user setup -  command: 'ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null root@{{ item }} echo root user is setup' -  register: result -  until: result.rc == 0 -  retries: 30 -  delay: 1 -  with_items: ips diff --git a/playbooks/libvirt/openshift-cluster/list.yml b/playbooks/libvirt/openshift-cluster/list.yml index 6bf07e3c6..25a25f791 100644 --- a/playbooks/libvirt/openshift-cluster/list.yml +++ b/playbooks/libvirt/openshift-cluster/list.yml @@ -1,43 +1,23 @@ +---  - name: Generate oo_list_hosts group    hosts: localhost -  connection: local    gather_facts: no - -  vars: -    libvirt_uri: 'qemu:///system' - +  vars_files: +  - vars.yml    tasks: -    - name: List VMs -      virt: -        command: list_vms -      register: list_vms - -    - name: Collect MAC addresses of the VMs -      shell: 'virsh -c {{ libvirt_uri }} dumpxml {{ item }} | xmllint --xpath "string(//domain/devices/interface/mac/@address)" -' -      register: scratch_mac -      with_items: '{{ list_vms.list_vms }}' -      when: item|truncate(cluster_id|length+1, True) == '{{ cluster_id }}-...' - -    - name: Collect IP addresses of the VMs -      shell: "awk '/{{ item.stdout }}/ {print $1}' /proc/net/arp" -      register: scratch_ip -      with_items: '{{ scratch_mac.results }}' -      when: item.skipped is not defined - -    - name: Add hosts -      add_host: -        hostname: '{{ item[0] }}' -        ansible_ssh_host: '{{ item[1].stdout }}' -        ansible_ssh_user: root -        groups: oo_list_hosts -      with_together: -        - '{{ list_vms.list_vms }}' -        - '{{ scratch_ip.results }}' -      when: item[1].skipped is not defined +  - set_fact: scratch_group=tag_env-{{ cluster_id }} +    when: cluster_id != '' +  - set_fact: scratch_group=all +    when: cluster_id == '' +  - add_host: +      name: "{{ item }}" +      groups: oo_list_hosts +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[scratch_group] | default([]) | difference(['localhost'])  - name: List Hosts    hosts: oo_list_hosts -    tasks: -    - debug: -        msg: 'public:{{ansible_default_ipv4.address}} private:{{ansible_default_ipv4.address}}' +  - debug: +      msg: 'public:{{ansible_default_ipv4.address}} private:{{ansible_default_ipv4.address}} deployment-type: {{ hostvars[inventory_hostname].group_names | oo_get_deployment_type_from_groups }}' diff --git a/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt.yml b/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt.yml new file mode 100644 index 000000000..f237c1a60 --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt.yml @@ -0,0 +1,6 @@ +--- +- include: configure_libvirt_storage_pool.yml +  when: libvirt_storage_pool is defined and libvirt_storage_pool_path is defined + +- include: configure_libvirt_network.yml +  when: libvirt_network is defined diff --git a/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt_network.yml b/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt_network.yml new file mode 100644 index 000000000..1cd83f7be --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt_network.yml @@ -0,0 +1,27 @@ +--- +- name: Test if libvirt network for openshift already exists +  command: "virsh -c {{ libvirt_uri }} net-info {{ libvirt_network }}" +  register: net_info_result +  changed_when: False +  failed_when: "net_info_result.rc != 0 and 'error: Network not found:' not in net_info_result.stderr" + +- name: Create a temp directory for the template xml file +  command: "/usr/bin/mktemp -d /tmp/openshift-ansible-XXXXXXX" +  register: mktemp +  when: net_info_result.rc == 1 + +- name: Create network xml file +  template: +    src: templates/network.xml +    dest: "{{ mktemp.stdout }}/network.xml" +  when: net_info_result.rc == 1 + +- name: Create libvirt network for openshift +  command: "virsh -c {{ libvirt_uri }} net-create {{ mktemp.stdout }}/network.xml" +  when: net_info_result.rc == 1 + +- name: Remove the temp directory +  file: +    path: "{{ mktemp.stdout }}" +    state: absent +  when: net_info_result.rc == 1 diff --git a/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt_storage_pool.yml b/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt_storage_pool.yml new file mode 100644 index 000000000..817acb250 --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/tasks/configure_libvirt_storage_pool.yml @@ -0,0 +1,27 @@ +--- +- name: Create libvirt storage directory for openshift +  file: +    dest: "{{ libvirt_storage_pool_path }}" +    state: directory + +- acl: +    default: yes +    entity: kvm +    etype: group +    name: "{{ libvirt_storage_pool_path }}" +    permissions: rwx +    state: present + +- name: Test if libvirt storage pool for openshift already exists +  command: "virsh -c {{ libvirt_uri }} pool-info {{ libvirt_storage_pool }}" +  register: pool_info_result +  changed_when: False +  failed_when: "pool_info_result.rc != 0 and 'error: Storage pool not found:' not in pool_info_result.stderr" + +- name: Create the libvirt storage pool for openshift +  command: 'virsh -c {{ libvirt_uri }} pool-create-as {{ libvirt_storage_pool }} dir --target {{ libvirt_storage_pool_path }}' +  when: pool_info_result.rc == 1 + +- name: Refresh the libvirt storage pool for openshift +  command: 'virsh -c {{ libvirt_uri }} pool-refresh {{ libvirt_storage_pool }}' +  when: pool_info_result.rc == 1 diff --git a/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml b/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml new file mode 100644 index 000000000..96d440096 --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/tasks/launch_instances.yml @@ -0,0 +1,104 @@ +--- +# TODO: Add support for choosing base image based on deployment_type and os +# wanted (os wanted needs support added in bin/cluster with sane defaults: +# fedora/centos for origin, rhel for online/enterprise) + +# TODO: create a role to encapsulate some of this complexity, possibly also +# create a module to manage the storage tasks, network tasks, and possibly +# even handle the libvirt tasks to set metadata in the domain xml and be able +# to create/query data about vms without having to use xml the python libvirt +# bindings look like a good candidate for this + +- name: Download Base Cloud image +  get_url: +    url: '{{ image_url }}' +    sha256sum: '{{ image_sha256 }}' +    dest: '{{ os_libvirt_storage_pool_path }}/{{ image_name }}' + +- name: Create the cloud-init config drive path +  file: +    dest: '{{ os_libvirt_storage_pool_path }}/{{ item }}_configdrive/' +    state: directory +  with_items: instances + +- name: Create the cloud-init config drive files +  template: +    src: '{{ item[1] }}' +    dest: '{{ os_libvirt_storage_pool_path }}/{{ item[0] }}_configdrive/{{ item[1] }}' +  with_nested: +    - instances +    - [ user-data, meta-data ] + +- name: Create the cloud-init config drive +  command: 'genisoimage -output {{ os_libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso -volid cidata -joliet -rock user-data meta-data' +  args: +    chdir: '{{ os_libvirt_storage_pool_path }}/{{ item }}_configdrive/' +    creates: '{{ os_libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso' +  with_items: instances + +- name: Create VMs drives +  command: 'virsh -c {{ libvirt_uri }} vol-create-as {{ os_libvirt_storage_pool }} {{ item }}.qcow2 10G --format qcow2 --backing-vol {{ image_name }} --backing-vol-format qcow2' +  with_items: instances + +- name: Create VMs +  virt: +    name: '{{ item }}' +    command: define +    xml: "{{ lookup('template', '../templates/domain.xml') }}" +    uri: '{{ libvirt_uri }}' +  with_items: instances + +- name: Start VMs +  virt: +    name: '{{ item }}' +    state: running +    uri: '{{ libvirt_uri }}' +  with_items: instances + +- name: Collect MAC addresses of the VMs +  shell: 'virsh -c {{ libvirt_uri }} dumpxml {{ item }} | xmllint --xpath "string(//domain/devices/interface/mac/@address)" -' +  register: scratch_mac +  with_items: instances + +- name: Wait for the VMs to get an IP +  command: "egrep -c '{{ scratch_mac.results | oo_collect('stdout') | join('|') }}' /proc/net/arp" +  ignore_errors: yes +  register: nb_allocated_ips +  until: nb_allocated_ips.stdout == '{{ instances | length }}' +  retries: 30 +  delay: 1 + +- name: Collect IP addresses of the VMs +  shell: "awk '/{{ item.stdout }}/ {print $1}' /proc/net/arp" +  register: scratch_ip +  with_items: scratch_mac.results + +- set_fact: +    ips: "{{ scratch_ip.results | oo_collect('stdout') }}" + +- name: Add new instances +  add_host: +    hostname: '{{ item.0 }}' +    ansible_ssh_host: '{{ item.1 }}' +    ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +    ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    groups: 'tag_env-{{ cluster }}, tag_host-type-{{ type }}, tag_env-host-type-{{ cluster }}-openshift-{{ type }}' +  with_together: +    - instances +    - ips + +- name: Wait for ssh +  wait_for: +    host: '{{ item }}' +    port: 22 +  with_items: ips + +- name: Wait for openshift user setup +  command: 'ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null openshift@{{ item.1 }} echo openshift user is setup' +  register: result +  until: result.rc == 0 +  retries: 30 +  delay: 1 +  with_together: +  - instances +  - ips diff --git a/playbooks/libvirt/templates/domain.xml b/playbooks/libvirt/openshift-cluster/templates/domain.xml index da037d138..8cb017367 100644 --- a/playbooks/libvirt/templates/domain.xml +++ b/playbooks/libvirt/openshift-cluster/templates/domain.xml @@ -1,6 +1,12 @@  <domain type='kvm' id='8'>    <name>{{ item }}</name>    <memory unit='GiB'>1</memory> +  <metadata xmlns:ansible="https://github.com/ansible/ansible"> +    <ansible:tag>deployment-type-{{ deployment_type }}</ansible:tag> +    <ansible:tag>env-{{ cluster }}</ansible:tag> +    <ansible:tag>env-host-type-{{ cluster }}-openshift-{{ type }}</ansible:tag> +    <ansible:tag>host-type-{{ type }}</ansible:tag> +  </metadata>    <currentMemory unit='GiB'>1</currentMemory>    <vcpu placement='static'>2</vcpu>    <os> @@ -24,18 +30,18 @@      <emulator>/usr/bin/qemu-system-x86_64</emulator>      <disk type='file' device='disk'>        <driver name='qemu' type='qcow2'/> -      <source file='{{ libvirt_storage_pool_path }}/{{ item }}.qcow2'/> +      <source file='{{ os_libvirt_storage_pool_path }}/{{ item }}.qcow2'/>        <target dev='vda' bus='virtio'/>      </disk>      <disk type='file' device='cdrom'>        <driver name='qemu' type='raw'/> -      <source file='{{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso'/> +      <source file='{{ os_libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso'/>        <target dev='vdb' bus='virtio'/>        <readonly/>      </disk>      <controller type='usb' index='0' />      <interface type='network'> -      <source network='default'/> +      <source network='{{ os_libvirt_network }}'/>        <model type='virtio'/>      </interface>      <serial type='pty'> @@ -49,7 +55,6 @@      </channel>      <input type='tablet' bus='usb' />      <input type='mouse' bus='ps2'/> -    <input type='keyboard' bus='ps2'/>      <graphics type='spice' autoport='yes' />      <video>        <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/> diff --git a/playbooks/libvirt/openshift-cluster/templates/meta-data b/playbooks/libvirt/openshift-cluster/templates/meta-data new file mode 100644 index 000000000..6b421770d --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/templates/meta-data @@ -0,0 +1,3 @@ +instance-id: {{ item[0] }} +hostname: {{ item[0] }} +local-hostname: {{ item[0] }}.example.com diff --git a/playbooks/libvirt/openshift-cluster/templates/network.xml b/playbooks/libvirt/openshift-cluster/templates/network.xml new file mode 100644 index 000000000..86dcd62bb --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/templates/network.xml @@ -0,0 +1,23 @@ +<network> +  <name>openshift-ansible</name> +  <forward mode='nat'> +    <nat> +      <port start='1024' end='65535'/> +    </nat> +  </forward> +  <!-- TODO: query for first available virbr interface available --> +  <bridge name='virbr3' stp='on' delay='0'/> +  <!-- TODO: make overridable --> +  <domain name='example.com'/> +  <dns> +    <!-- TODO: automatically add host entries --> +  </dns> +  <!-- TODO: query for available address space --> +  <ip address='192.168.55.1' netmask='255.255.255.0'> +    <dhcp> +      <range start='192.168.55.2' end='192.168.55.254'/> +      <!-- TODO: add static entries addresses for the hosts to be created --> +    </dhcp> +  </ip> +</network> + diff --git a/playbooks/libvirt/openshift-cluster/templates/user-data b/playbooks/libvirt/openshift-cluster/templates/user-data new file mode 100644 index 000000000..77b788109 --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/templates/user-data @@ -0,0 +1,23 @@ +#cloud-config +disable_root: true + +hostname: {{ item[0] }} +fqdn: {{ item[0] }}.example.com +manage_etc_hosts: true + +users: +  - default +  - name: root +    ssh_authorized_keys: +    - {{ lookup('file', '~/.ssh/id_rsa.pub') }} + +system_info: +  default_user: +    name: openshift +    sudo: ["ALL=(ALL) NOPASSWD: ALL"] + +ssh_authorized_keys: +  - {{ lookup('file', '~/.ssh/id_rsa.pub') }} + +bootcmd: +  - NETWORK_CONFIG=/etc/sysconfig/network-scripts/ifcfg-eth0; if ! grep DHCP_HOSTNAME ${NETWORK_CONFIG}; then echo 'DHCP_HOSTNAME="{{ item[0] }}.example.com"' >> ${NETWORK_CONFIG}; fi; pkill -9 dhclient; service network restart diff --git a/playbooks/libvirt/openshift-cluster/terminate.yml b/playbooks/libvirt/openshift-cluster/terminate.yml index c609169d3..b173a09dd 100644 --- a/playbooks/libvirt/openshift-cluster/terminate.yml +++ b/playbooks/libvirt/openshift-cluster/terminate.yml @@ -1,41 +1,44 @@ +--- +# TODO: does not handle a non-existant cluster gracefully +  - name: Terminate instance(s)    hosts: localhost -  connection: local    gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - set_fact: cluster_group=tag_env-{{ cluster_id }} +  - add_host: +      name: "{{ item }}" +      groups: oo_hosts_to_terminate +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups[cluster_group] | default([]) -  vars: -    libvirt_storage_pool_path: "{{ lookup('env','HOME') }}/libvirt-storage-pool-openshift" -    libvirt_storage_pool: 'openshift' -    libvirt_uri: 'qemu:///system' +  - name: Destroy VMs +    virt: +      name: '{{ item[0] }}' +      command: '{{ item[1] }}' +      uri: '{{ libvirt_uri }}' +    with_nested: +    - groups['oo_hosts_to_terminate'] +    - [ destroy, undefine ] -  tasks: -    - name: List VMs -      virt: -        command: list_vms -      register: list_vms +  - name: Delete VMs drives +    command: 'virsh -c {{ libvirt_uri }} vol-delete --pool {{ libvirt_storage_pool }} {{ item }}.qcow2' +    args: +      removes: '{{ libvirt_storage_pool_path }}/{{ item }}.qcow2' +    with_items: groups['oo_hosts_to_terminate'] -    - name: Destroy VMs -      virt: -        name: '{{ item[0] }}' -        command: '{{ item[1] }}' -        uri: '{{ libvirt_uri }}' -      with_nested: -        - '{{ list_vms.list_vms }}' -        - [ destroy, undefine ] -      when: item[0]|truncate(cluster_id|length+1, True) == '{{ cluster_id }}-...' +  - name: Delete the VM cloud-init image +    file: +      path: '{{ libvirt_storage_pool_path }}/{{ item }}_cloud-init.iso' +      state: absent +    with_items: groups['oo_hosts_to_terminate'] -    - name: Delete VMs config drive -      file: -        path: '{{ libvirt_storage_pool_path }}/{{ item }}_configdrive/openstack' -        state: absent -      with_items: '{{ list_vms.list_vms }}' -      when: item|truncate(cluster_id|length+1, True) == '{{ cluster_id }}-...' +  - name: Remove the cloud-init config directory +    file: +      path: '{{ libvirt_storage_pool_path }}/{{ item }}_configdrive/' +      state: absent +    with_items: groups['oo_hosts_to_terminate'] -    - name: Delete VMs drives -      command: 'virsh -c {{ libvirt_uri }} vol-delete --pool {{ libvirt_storage_pool }} {{ item[0] }}{{ item[1] }}' -      args: -        removes: '{{ libvirt_storage_pool_path }}/{{ item[0] }}{{ item[1] }}' -      with_nested: -        - '{{ list_vms.list_vms }}' -        - [ '_configdrive', '_cloud-init.iso', '.qcow2' ] -      when: item[0]|truncate(cluster_id|length+1, True) == '{{ cluster_id }}-...' diff --git a/playbooks/libvirt/openshift-cluster/update.yml b/playbooks/libvirt/openshift-cluster/update.yml new file mode 100644 index 000000000..57e36db9e --- /dev/null +++ b/playbooks/libvirt/openshift-cluster/update.yml @@ -0,0 +1,18 @@ +--- +- name: Populate oo_hosts_to_update group +  hosts: localhost +  gather_facts: no +  vars_files: +  - vars.yml +  tasks: +  - name: Evaluate oo_hosts_to_update +    add_host: +      name: "{{ item }}" +      groups: oo_hosts_to_update +      ansible_ssh_user: "{{ deployment_vars[deployment_type].ssh_user }}" +      ansible_sudo: "{{ deployment_vars[deployment_type].sudo }}" +    with_items: groups["tag_env-host-type-{{ cluster_id }}-openshift-master"] | union(groups["tag_env-host-type-{{ cluster_id }}-openshift-node"]) | default([]) + +- include: ../../common/openshift-cluster/update_repos_and_packages.yml + +- include: config.yml diff --git a/playbooks/libvirt/openshift-cluster/vars.yml b/playbooks/libvirt/openshift-cluster/vars.yml index 4e4eecd46..65d954fee 100644 --- a/playbooks/libvirt/openshift-cluster/vars.yml +++ b/playbooks/libvirt/openshift-cluster/vars.yml @@ -1,7 +1,33 @@ -# base_image_url: http://download.fedoraproject.org/pub/fedora/linux/releases/21/Cloud/Images/x86_64/Fedora-Cloud-Base-20141203-21.x86_64.qcow2 -# base_image_name: Fedora-Cloud-Base-20141203-21.x86_64.qcow2 -# base_image_sha256: 3a99bb89f33e3d4ee826c8160053cdb8a72c80cd23350b776ce73cd244467d86 +--- +libvirt_storage_pool_path: "{{ lookup('env','HOME') }}/libvirt-storage-pool-openshift-ansible" +libvirt_storage_pool: 'openshift-ansible' +libvirt_network: openshift-ansible +libvirt_uri: 'qemu:///system' -base_image_url: http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 -base_image_name: CentOS-7-x86_64-GenericCloud.qcow2 -base_image_sha256: e324e3ab1d24a1bbf035ddb365e7f9058c0b454acf48d7aa15c5519fae5998ab +deployment_vars: +  origin: +    image: +      url: "http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2" +      name: CentOS-7-x86_64-GenericCloud.qcow2 +      sha256: e324e3ab1d24a1bbf035ddb365e7f9058c0b454acf48d7aa15c5519fae5998ab +    ssh_user: openshift +    sudo: yes +  online: +    image: +      url: +      name: +      sha256: +    ssh_user: root +    sudo: no +  enterprise: +    image: +      url: +      name: +      sha256: +    ssh_user: openshift +    sudo: yes +#  origin: +#    fedora: +#      url: "http://download.fedoraproject.org/pub/fedora/linux/releases/21/Cloud/Images/x86_64/Fedora-Cloud-Base-20141203-21.x86_64.qcow2" +#      name: Fedora-Cloud-Base-20141203-21.x86_64.qcow2 +#      sha256: 3a99bb89f33e3d4ee826c8160053cdb8a72c80cd23350b776ce73cd244467d86 diff --git a/playbooks/libvirt/openshift-master/config.yml b/playbooks/libvirt/openshift-master/config.yml deleted file mode 100644 index dd95fd57f..000000000 --- a/playbooks/libvirt/openshift-master/config.yml +++ /dev/null @@ -1,21 +0,0 @@ -- name: master/config.yml, populate oo_masters_to_config host group if needed -  hosts: localhost -  gather_facts: no -  tasks: -    - name: "Evaluate oo_host_group_exp if it's set" -      add_host: -        name: '{{ item }}' -        groups: oo_masters_to_config -      with_items: "{{ oo_host_group_exp | default('') }}" -      when: oo_host_group_exp is defined - -- name: Configure instances -  hosts: oo_masters_to_config -  vars: -    openshift_hostname: '{{ ansible_default_ipv4.address }}' -  vars_files: -    - vars.yml -  roles: -    - openshift_master -    - pods -    - os_env_extras diff --git a/playbooks/libvirt/openshift-master/vars.yml b/playbooks/libvirt/openshift-master/vars.yml deleted file mode 100644 index ad0c0fbe2..000000000 --- a/playbooks/libvirt/openshift-master/vars.yml +++ /dev/null @@ -1 +0,0 @@ -openshift_debug_level: 4 diff --git a/playbooks/libvirt/openshift-node/config.yml b/playbooks/libvirt/openshift-node/config.yml deleted file mode 100644 index 3244a8046..000000000 --- a/playbooks/libvirt/openshift-node/config.yml +++ /dev/null @@ -1,102 +0,0 @@ -- name: node/config.yml, populate oo_nodes_to_config host group if needed -  hosts: localhost -  gather_facts: no -  tasks: -    - name: "Evaluate oo_host_group_exp if it's set" -      add_host: -        name: '{{ item }}' -        groups: oo_nodes_to_config -      with_items: "{{ oo_host_group_exp | default('') }}" -      when: oo_host_group_exp is defined - -    - add_host: -        name: "{{ groups['tag_env-host-type-' ~ cluster_id ~ '-openshift-master'][0] }}" -        groups: oo_first_master -      when: oo_host_group_exp is defined - - -- name: Gather and set facts for hosts to configure -  hosts: oo_nodes_to_config -  roles: -  - openshift_facts -  tasks: -  # Since the master is registering the nodes before they are configured, we -  # need to make sure to set the node properties beforehand if we do not want -  # the defaults -  - openshift_facts: -      role: "{{ item.role }}" -      local_facts: "{{ item.local_facts }}" -    with_items: -    - role: common -      local_facts: -        hostname: "{{ ansible_default_ipv4.address }}" -    - role: node -      local_facts: -        external_id: "{{ openshift_node_external_id | default(None) }}" -        resources_cpu: "{{ openshfit_node_resources_cpu | default(None) }}" -        resources_memory: "{{ openshfit_node_resources_memory | default(None) }}" -        pod_cidr: "{{ openshfit_node_pod_cidr | default(None) }}" -        labels: "{{ openshfit_node_labels | default(None) }}" -        annotations: "{{ openshfit_node_annotations | default(None) }}" - - -- name: Register nodes -  hosts: oo_first_master -  vars: -    openshift_nodes: "{{ hostvars -          | oo_select_keys(groups['oo_nodes_to_config']) }}" -  roles: -  - openshift_register_nodes -  tasks: -  - name: Create local temp directory for syncing certs -    local_action: command /usr/bin/mktemp -d /tmp/openshift-ansible-XXXXXXX -    register: mktemp - -  - name: Sync master certs to localhost -    synchronize: -      mode: pull -      checksum: yes -      src: /var/lib/openshift/openshift.local.certificates -      dest: "{{ mktemp.stdout }}" - -- name: Configure instances -  hosts: oo_nodes_to_config -  vars_files: -  - vars.yml -  vars: -    sync_tmpdir: "{{ hostvars[groups['oo_first_master'][0]].mktemp.stdout }}" -    cert_parent_rel_path: openshift.local.certificates -    cert_rel_path: "{{ cert_parent_rel_path }}/node-{{ openshift.common.hostname }}" -    cert_base_path: /var/lib/openshift -    cert_parent_path: "{{ cert_base_path }}/{{ cert_parent_rel_path }}" -    cert_path: "{{ cert_base_path }}/{{ cert_rel_path }}" -  pre_tasks: -  - name: Ensure certificate directories exists -    file: -      path: "{{ item }}" -      state: directory -    with_items: -    - "{{ cert_path }}" -    - "{{ cert_parent_path }}/ca" - -  # TODO: notify restart openshift-node and/or restart openshift-sdn-node, -  # possibly test service started time against certificate/config file -  # timestamps in openshift-node or openshift-sdn-node to trigger notify -  - name: Sync certs to nodes -    synchronize: -      checksum: yes -      src: "{{ item.src }}" -      dest: "{{ item.dest }}" -      owner: no -      group: no -    with_items: -    - src: "{{ sync_tmpdir }}/{{ cert_rel_path }}" -      dest: "{{ cert_parent_path }}" -    - src: "{{ sync_tmpdir }}/{{ cert_parent_rel_path }}/ca/cert.crt" -      dest: "{{ cert_parent_path }}/ca/cert.crt" -  - local_action: file name={{ sync_tmpdir }} state=absent -    run_once: true -  roles: -    - openshift_node -    - os_env_extras -    - os_env_extras_node diff --git a/playbooks/libvirt/openshift-node/roles b/playbooks/libvirt/openshift-node/roles deleted file mode 120000 index 20c4c58cf..000000000 --- a/playbooks/libvirt/openshift-node/roles +++ /dev/null @@ -1 +0,0 @@ -../../../roles
\ No newline at end of file diff --git a/playbooks/libvirt/openshift-node/vars.yml b/playbooks/libvirt/openshift-node/vars.yml deleted file mode 100644 index ad0c0fbe2..000000000 --- a/playbooks/libvirt/openshift-node/vars.yml +++ /dev/null @@ -1 +0,0 @@ -openshift_debug_level: 4 diff --git a/playbooks/libvirt/templates/meta-data b/playbooks/libvirt/templates/meta-data deleted file mode 100644 index 5d779519f..000000000 --- a/playbooks/libvirt/templates/meta-data +++ /dev/null @@ -1,2 +0,0 @@ -instance-id: {{ item[0] }} -local-hostname: {{ item[0] }} diff --git a/playbooks/libvirt/templates/user-data b/playbooks/libvirt/templates/user-data deleted file mode 100644 index 985badc8e..000000000 --- a/playbooks/libvirt/templates/user-data +++ /dev/null @@ -1,10 +0,0 @@ -#cloud-config - -disable_root: 0 - -system_info: -  default_user: -    name: root - -ssh_authorized_keys: -  - {{ lookup('file', '~/.ssh/id_rsa.pub') }} diff --git a/roles/openshift_common/tasks/main.yml b/roles/openshift_common/tasks/main.yml index 941190534..c55677c3f 100644 --- a/roles/openshift_common/tasks/main.yml +++ b/roles/openshift_common/tasks/main.yml @@ -1,7 +1,7 @@  ---  - name: Set common OpenShift facts    openshift_facts: -    role: 'common' +    role: common      local_facts:        cluster_id: "{{ openshift_cluster_id | default('default') }}"        debug_level: "{{ openshift_debug_level | default(0) }}" @@ -10,7 +10,7 @@        public_hostname: "{{ openshift_public_hostname | default(None) }}"        public_ip: "{{ openshift_public_ip | default(None) }}"        use_openshift_sdn: "{{ openshift_use_openshift_sdn | default(None) }}" - +      deployment_type: "{{ openshift_deployment_type }}"  - name: Set hostname    hostname: name={{ openshift.common.hostname }} diff --git a/roles/openshift_common/vars/main.yml b/roles/openshift_common/vars/main.yml index 50816d319..9f657a2c7 100644 --- a/roles/openshift_common/vars/main.yml +++ b/roles/openshift_common/vars/main.yml @@ -5,3 +5,7 @@  # chains with the public zone (or the zone associated with the correct  # interfaces)  os_firewall_use_firewalld: False + +openshift_cert_parent_dir: /var/lib/openshift +openshift_cert_relative_dir: openshift.local.certificates +openshift_cert_dir: "{{ openshift_cert_parent_dir }}/{{ openshift_cert_relative_dir }}" diff --git a/roles/openshift_facts/library/openshift_facts.py b/roles/openshift_facts/library/openshift_facts.py index 0dd343443..1e0d5c605 100755 --- a/roles/openshift_facts/library/openshift_facts.py +++ b/roles/openshift_facts/library/openshift_facts.py @@ -21,8 +21,11 @@ class OpenShiftFactsUnsupportedRoleError(Exception):  class OpenShiftFactsFileWriteError(Exception):      pass +class OpenShiftFactsMetadataUnavailableError(Exception): +    pass +  class OpenShiftFacts(): -    known_roles = ['common', 'master', 'node', 'master_sdn', 'node_sdn'] +    known_roles = ['common', 'master', 'node', 'master_sdn', 'node_sdn', 'dns']      def __init__(self, role, filename, local_facts):          self.changed = False @@ -169,20 +172,18 @@ class OpenShiftFacts():          return hostname      def get_defaults(self, roles): -        hardware_facts = self.get_hardware_facts() -        net_facts = self.get_net_facts() -        base_facts = self.get_base_facts() +        ansible_facts = self.get_ansible_facts()          defaults = dict()          common = dict(use_openshift_sdn=True) -        ip = net_facts['default_ipv4']['address'] +        ip = ansible_facts['default_ipv4']['address']          common['ip'] = ip          common['public_ip'] = ip          rc, output, error = module.run_command(['hostname', '-f'])          hostname_f = output.strip() if rc == 0 else '' -        hostname_values = [hostname_f, base_facts['nodename'], base_facts['fqdn']] +        hostname_values = [hostname_f, ansible_facts['nodename'], ansible_facts['fqdn']]          hostname = self.choose_hostname(hostname_values)          common['hostname'] = hostname @@ -196,14 +197,14 @@ class OpenShiftFacts():              master = dict(api_use_ssl=True, api_port='8443',                      console_use_ssl=True, console_path='/console',                      console_port='8443', etcd_use_ssl=False, -                    etcd_port='4001') +                    etcd_port='4001', portal_net='172.30.17.0/24')              defaults['master'] = master          if 'node' in roles:              node = dict(external_id=common['hostname'], pod_cidr='',                          labels={}, annotations={}) -            node['resources_cpu'] = hardware_facts['processor_cores'] -            node['resources_memory'] = int(int(hardware_facts['memtotal_mb']) * 1024 * 1024 * 0.75) +            node['resources_cpu'] = ansible_facts['processor_cores'] +            node['resources_memory'] = int(int(ansible_facts['memtotal_mb']) * 1024 * 1024 * 0.75)              defaults['node'] = node          return defaults @@ -226,8 +227,7 @@ class OpenShiftFacts():      def query_metadata(self, metadata_url, headers=None, expect_json=False):          r, info = fetch_url(module, metadata_url, headers=headers)          if info['status'] != 200: -            module.fail_json(msg='Failed to query metadata', result=r, -                             info=info) +            raise OpenShiftFactsMetadataUnavailableError("Metadata unavailable")          if expect_json:              return module.from_json(r.read())          else: @@ -252,40 +252,27 @@ class OpenShiftFacts():      def get_provider_metadata(self, metadata_url, supports_recursive=False,                            headers=None, expect_json=False): -        if supports_recursive: -            metadata = self.query_metadata(metadata_url, headers, expect_json) -        else: -            metadata = self.walk_metadata(metadata_url, headers, expect_json) +        try: +            if supports_recursive: +                metadata = self.query_metadata(metadata_url, headers, expect_json) +            else: +                metadata = self.walk_metadata(metadata_url, headers, expect_json) +        except OpenShiftFactsMetadataUnavailableError as e: +            metadata = None          return metadata -    def get_hardware_facts(self): -        if not hasattr(self, 'hardware_facts'): -            self.hardware_facts = Hardware().populate() -        return self.hardware_facts - -    def get_base_facts(self): -        if not hasattr(self, 'base_facts'): -            self.base_facts = Facts().populate() -        return self.base_facts - -    def get_virt_facts(self): -        if not hasattr(self, 'virt_facts'): -            self.virt_facts = Virtual().populate() -        return self.virt_facts - -    def get_net_facts(self): -        if not hasattr(self, 'net_facts'): -            self.net_facts = Network(module).populate() -        return self.net_facts +    def get_ansible_facts(self): +        if not hasattr(self, 'ansible_facts'): +            self.ansible_facts = ansible_facts(module) +        return self.ansible_facts      def guess_host_provider(self):          # TODO: cloud provider facts should probably be submitted upstream -        virt_facts = self.get_virt_facts() -        hardware_facts = self.get_hardware_facts() -        product_name = hardware_facts['product_name'] -        product_version = hardware_facts['product_version'] -        virt_type = virt_facts['virtualization_type'] -        virt_role = virt_facts['virtualization_role'] +        ansible_facts = self.get_ansible_facts() +        product_name = ansible_facts['product_name'] +        product_version = ansible_facts['product_version'] +        virt_type = ansible_facts['virtualization_type'] +        virt_role = ansible_facts['virtualization_role']          provider = None          metadata = None @@ -300,8 +287,9 @@ class OpenShiftFacts():                                                    True)              # Filter sshKeys and serviceAccounts from gce metadata -            metadata['project']['attributes'].pop('sshKeys', None) -            metadata['instance'].pop('serviceAccounts', None) +            if metadata: +                metadata['project']['attributes'].pop('sshKeys', None) +                metadata['instance'].pop('serviceAccounts', None)          elif virt_type == 'xen' and virt_role == 'guest' and re.match(r'.*\.amazon$', product_version):              provider = 'ec2'              metadata_url = 'http://169.254.169.254/latest/meta-data/' @@ -310,12 +298,18 @@ class OpenShiftFacts():              provider = 'openstack'              metadata_url = 'http://169.254.169.254/openstack/latest/meta_data.json'              metadata = self.get_provider_metadata(metadata_url, True, None, True) -            ec2_compat_url = 'http://169.254.169.254/latest/meta-data/' -            metadata['ec2_compat'] = self.get_provider_metadata(ec2_compat_url) -            # Filter public_keys  and random_seed from openstack metadata -            metadata.pop('public_keys', None) -            metadata.pop('random_seed', None) +            if metadata: +                ec2_compat_url = 'http://169.254.169.254/latest/meta-data/' +                metadata['ec2_compat'] = self.get_provider_metadata(ec2_compat_url) + +                # Filter public_keys  and random_seed from openstack metadata +                metadata.pop('public_keys', None) +                metadata.pop('random_seed', None) + +                if not metadata['ec2_compat']: +                    metadata = None +          return dict(name=provider, metadata=metadata)      def normalize_provider_facts(self, provider, metadata): @@ -479,4 +473,6 @@ def main():  from ansible.module_utils.basic import *  from ansible.module_utils.facts import *  from ansible.module_utils.urls import * -main() + +if __name__ == '__main__': +    main() diff --git a/roles/openshift_master/tasks/main.yml b/roles/openshift_master/tasks/main.yml index aa615df39..1b1210007 100644 --- a/roles/openshift_master/tasks/main.yml +++ b/roles/openshift_master/tasks/main.yml @@ -11,33 +11,67 @@        api_url: "{{ openshift_master_api_url | default(None) }}"        api_use_ssl: "{{ openshift_master_api_use_ssl | default(None) }}"        public_api_url: "{{ openshift_master_public_api_url | default(None) }}" +      console_path: "{{ openshift_master_console_path | default(None) }}"        console_port: "{{ openshift_master_console_port | default(None) }}"        console_url: "{{ openshift_master_console_url | default(None) }}"        console_use_ssl: "{{ openshift_master_console_use_ssl | default(None) }}"        public_console_url: "{{ openshift_master_public_console_url | default(None) }}" +      etcd_port: "{{ openshift_master_etcd_port | default(None) }}"        etcd_use_ssl: "{{ openshift_master_etcd_use_ssl | default(None) }}" +      portal_net: "{{ openshift_master_portal_net | default(None) }}" + +# TODO: These values need to be configurable +- name: Set dns OpenShift facts +  openshift_facts: +    role: 'dns' +    local_facts: +      ip: "{{ openshift.common.ip }}" +      domain: local  - name: Install OpenShift Master package    yum: pkg=openshift-master state=installed +  register: install_result + +- name: Reload systemd units +  command: systemctl daemon-reload +  when: install_result | changed + +- name: Create certificate parent directory if it doesn't exist +  file: +    path: "{{ openshift_cert_parent_dir }}" +    state: directory + +- name: Create config parent directory if it doesn't exist +  file: +    path: "{{ openshift_master_config | dirname }}" +    state: directory + +# TODO: should probably use a template lookup for this +# TODO: should allow for setting --etcd, --kubernetes options +# TODO: recreate config if values change +- name: Use enterprise default for openshift_registry_url if not set +  set_fact: +    openshift_registry_url: "openshift3_beta/ose-${component}:${version}" +  when: openshift.common.deployment_type == 'enterprise' and openshift_registry_url is not defined +- name: Create master config +  command: > +    /usr/bin/openshift start master --write-config +    --config={{ openshift_master_config }} +    --portal-net={{ openshift.master.portal_net }} +    --master={{ openshift.master.api_url }} +    --public-master={{ openshift.master.public_api_url }} +    --listen={{ 'https' if openshift.master.api_use_ssl else 'http' }}://0.0.0.0:{{ openshift.master.api_port }} +    {{ ('--images=' ~ openshift_registry_url) if openshift_registry_url is defined else '' }} +    {{ ('--nodes=' ~ openshift_node_ips | join(',')) if openshift_node_ips is defined else '' }} +  args: +    chdir: "{{ openshift_cert_parent_dir }}" +    creates: "{{ openshift_master_config }}" -# TODO: We should pre-generate the master config and point to the generated -# config rather than setting command line flags here  - name: Configure OpenShift settings    lineinfile:      dest: /etc/sysconfig/openshift-master      regexp: '^OPTIONS=' -    line: "OPTIONS=\"--master={{ openshift.common.hostname }} --public-master={{ openshift.common.public_hostname }} {% if openshift_node_ips %} --nodes={{ openshift_node_ips | join(',') }} {% endif %} --loglevel={{ openshift.master.debug_level }}\"" -  notify: -  - restart openshift-master - -# TODO: should this be populated by a fact based on the deployment type -# (origin, online, enterprise)? -- name: Set default registry url -  lineinfile: -    dest: /etc/sysconfig/openshift-master -    regexp: '^IMAGES=' -    line: "IMAGES={{ openshift_registry_url }}" -  when: openshift_registry_url is defined +    line: "OPTIONS=\"--config={{ openshift_master_config }} --loglevel={{ openshift.master.debug_level }}\""    notify:    - restart openshift-master @@ -53,6 +87,6 @@  # TODO: Update this file if the contents of the source file are not present in  # the dest file, will need to make sure to ignore things that could be added  - name: Configure root user kubeconfig -  command: cp /var/lib/openshift/openshift.local.certificates/openshift-client/.kubeconfig /root/.kube/.kubeconfig +  command: cp {{ openshift_cert_dir }}/openshift-client/.kubeconfig /root/.kube/.kubeconfig    args:      creates: /root/.kube/.kubeconfig diff --git a/roles/openshift_master/vars/main.yml b/roles/openshift_master/vars/main.yml new file mode 100644 index 000000000..c52d957ac --- /dev/null +++ b/roles/openshift_master/vars/main.yml @@ -0,0 +1,5 @@ +--- +openshift_master_config: /etc/openshift/master.yaml +openshift_master_ca_dir: "{{ openshift_cert_dir }}/ca" +openshift_master_ca_cert: "{{ openshift_master_ca_dir }}/cert.crt" +openshift_master_ca_key: "{{ openshift_master_ca_dir }}/key.key" diff --git a/roles/openshift_node/tasks/main.yml b/roles/openshift_node/tasks/main.yml index e3c04585b..3d56bdd67 100644 --- a/roles/openshift_node/tasks/main.yml +++ b/roles/openshift_node/tasks/main.yml @@ -13,17 +13,22 @@    failed_when: not result.stat.exists    register: result    with_items: -  - "{{ cert_path }}" -  - "{{ cert_path }}/cert.crt" -  - "{{ cert_path }}/key.key" -  - "{{ cert_path }}/.kubeconfig" -  - "{{ cert_path }}/server.crt" -  - "{{ cert_path }}/server.key" -  - "{{ cert_parent_path }}/ca/cert.crt" -  #- "{{ cert_path }}/node.yaml" +  - "{{ openshift_node_cert_dir }}" +  - "{{ openshift_node_cert_dir }}/ca.crt" +  - "{{ openshift_node_cert_dir }}/client.crt" +  - "{{ openshift_node_cert_dir }}/client.key" +  - "{{ openshift_node_cert_dir }}/.kubeconfig" +  - "{{ openshift_node_cert_dir }}/node-config.yaml" +  - "{{ openshift_node_cert_dir }}/server.crt" +  - "{{ openshift_node_cert_dir }}/server.key"  - name: Install OpenShift Node package    yum: pkg=openshift-node state=installed +  register: install_result + +- name: Reload systemd units +  command: systemctl daemon-reload +  when: install_result | changed  # --create-certs=false is a temporary workaround until  # https://github.com/openshift/origin/pull/1361 is merged upstream and it is @@ -32,16 +37,7 @@    lineinfile:      dest: /etc/sysconfig/openshift-node      regexp: '^OPTIONS=' -    line: "OPTIONS=\"--hostname={{ openshift.common.hostname }} --loglevel={{ openshift.node.debug_level }} --create-certs=false\"" -  notify: -  - restart openshift-node - -- name: Set default registry url -  lineinfile: -    dest: /etc/sysconfig/openshift-node -    regexp: '^IMAGES=' -    line: "IMAGES={{ openshift_registry_url }}" -  when: openshift_registry_url is defined +    line: "OPTIONS=\"--loglevel={{ openshift.node.debug_level }} --config={{ openshift_node_cert_dir }}/node-config.yaml\""    notify:    - restart openshift-node diff --git a/roles/openshift_node/vars/main.yml b/roles/openshift_node/vars/main.yml new file mode 100644 index 000000000..c6be83139 --- /dev/null +++ b/roles/openshift_node/vars/main.yml @@ -0,0 +1,2 @@ +--- +openshift_node_cert_dir: /etc/openshift/node diff --git a/roles/openshift_register_nodes/defaults/main.yml b/roles/openshift_register_nodes/defaults/main.yml index 3501e8922..a0befab44 100644 --- a/roles/openshift_register_nodes/defaults/main.yml +++ b/roles/openshift_register_nodes/defaults/main.yml @@ -1,5 +1,2 @@  ---  openshift_kube_api_version: v1beta1 -openshift_cert_dir: openshift.local.certificates -openshift_cert_dir_parent: /var/lib/openshift -openshift_cert_dir_abs: "{{ openshift_cert_dir_parent ~ '/' ~ openshift_cert_dir }}" diff --git a/roles/openshift_register_nodes/library/kubernetes_register_node.py b/roles/openshift_register_nodes/library/kubernetes_register_node.py index 8ebeb087a..1ec977716 100755 --- a/roles/openshift_register_nodes/library/kubernetes_register_node.py +++ b/roles/openshift_register_nodes/library/kubernetes_register_node.py @@ -97,10 +97,8 @@ class ClientConfigException(Exception):  class ClientConfig:      def __init__(self, client_opts, module): -        _, output, error = module.run_command(["/usr/bin/openshift", "ex", -                                               "config", "view", "-o", -                                               "json"] + client_opts, -                                              check_rc = True) +        kubectl = module.params['kubectl_cmd'] +        _, output, error = module.run_command(kubectl + ["config", "view", "-o", "json"] + client_opts, check_rc = True)          self.config = json.loads(output)          if not (bool(self.config['clusters']) or @@ -146,6 +144,9 @@ class ClientConfig:      def get_cluster_for_context(self, context):          return self.get_value_for_context(context, 'cluster') +    def get_namespace_for_context(self, context): +        return self.get_value_for_context(context, 'namespace') +  class Util:      @staticmethod      def remove_empty_elements(mapping): @@ -247,15 +248,15 @@ class Node:          return Util.remove_empty_elements(node)      def exists(self): -        _, output, error = self.module.run_command(["/usr/bin/osc", "get", -                                                    "nodes"] +  self.client_opts, -                                                   check_rc = True) +        kubectl = self.module.params['kubectl_cmd'] +        _, output, error = self.module.run_command(kubectl + ["get", "nodes"] +  self.client_opts, check_rc = True)          if re.search(self.module.params['name'], output, re.MULTILINE):              return True          return False      def create(self): -        cmd = ['/usr/bin/osc'] + self.client_opts + ['create', 'node', '-f', '-'] +        kubectl = self.module.params['kubectl_cmd'] +        cmd = kubectl + self.client_opts + ['create', '-f', '-']          rc, output, error = self.module.run_command(cmd,                                                 data=self.module.jsonify(self.get_node()))          if rc != 0: @@ -273,24 +274,26 @@ class Node:  def main():      module = AnsibleModule( -        argument_spec      = dict( -            name           = dict(required = True, type = 'str'), -            host_ip        = dict(type = 'str'), -            hostnames      = dict(type = 'list', default = []), -            external_ips   = dict(type = 'list', default = []), -            internal_ips   = dict(type = 'list', default = []), -            api_version    = dict(type = 'str', default = 'v1beta1', # TODO: after kube rebase, we can default to v1beta3 -                                  choices = ['v1beta1', 'v1beta3']), -            cpu            = dict(type = 'str'), -            memory         = dict(type = 'str'), -            labels         = dict(type = 'dict', default = {}), # TODO: needs documented -            annotations    = dict(type = 'dict', default = {}), # TODO: needs documented -            pod_cidr       = dict(type = 'str'), # TODO: needs documented -            external_id    = dict(type = 'str'), # TODO: needs documented -            client_config  = dict(type = 'str'), # TODO: needs documented -            client_cluster = dict(type = 'str', default = 'master'), # TODO: needs documented -            client_context = dict(type = 'str', default = 'master'), # TODO: needs documented -            client_user    = dict(type = 'str', default = 'admin') # TODO: needs documented +        argument_spec        = dict( +            name             = dict(required = True, type = 'str'), +            host_ip          = dict(type = 'str'), +            hostnames        = dict(type = 'list', default = []), +            external_ips     = dict(type = 'list', default = []), +            internal_ips     = dict(type = 'list', default = []), +            api_version      = dict(type = 'str', default = 'v1beta1', # TODO: after kube rebase, we can default to v1beta3 +                                    choices = ['v1beta1', 'v1beta3']), +            cpu              = dict(type = 'str'), +            memory           = dict(type = 'str'), +            labels           = dict(type = 'dict', default = {}), # TODO: needs documented +            annotations      = dict(type = 'dict', default = {}), # TODO: needs documented +            pod_cidr         = dict(type = 'str'), # TODO: needs documented +            external_id      = dict(type = 'str'), # TODO: needs documented +            client_config    = dict(type = 'str'), # TODO: needs documented +            client_cluster   = dict(type = 'str', default = 'master'), # TODO: needs documented +            client_context   = dict(type = 'str', default = 'default'), # TODO: needs documented +            client_namespace = dict(type = 'str', default = 'default'), # TODO: needs documented +            client_user      = dict(type = 'str', default = 'system:openshift-client'), # TODO: needs documented +            kubectl_cmd      = dict(type = 'list', default = ['kubectl']) # TODO: needs documented          ),          mutually_exclusive = [              ['host_ip', 'external_ips'], @@ -333,14 +336,16 @@ def main():      client_cluster = module.params['client_cluster']      if config.has_cluster(client_cluster): -        if client_cluster != config.get_cluster_for_context(client_cluster): +        if client_cluster != config.get_cluster_for_context(client_context):              client_opts.append("--cluster=%s" % client_cluster)      else:          module.fail_json(msg="Cluster %s not found in client config" %                           client_cluster) -    # TODO: provide sane defaults for some (like hostname, externalIP, -    # internalIP, etc) +    client_namespace = module.params['client_namespace'] +    if client_namespace != config.get_namespace_for_context(client_context): +        client_opts.append("--namespace=%s" % client_namespace) +      node = Node(module, client_opts, module.params['api_version'],                  module.params['name'], module.params['host_ip'],                  module.params['hostnames'], module.params['external_ips'], diff --git a/roles/openshift_register_nodes/tasks/main.yml b/roles/openshift_register_nodes/tasks/main.yml index 7319b88b1..85f490f70 100644 --- a/roles/openshift_register_nodes/tasks/main.yml +++ b/roles/openshift_register_nodes/tasks/main.yml @@ -3,53 +3,37 @@  # TODO: recreate master/node configs if settings that affect the configs  # change (hostname, public_hostname, ip, public_ip, etc) -# TODO: create a failed_when condition -- name: Create node server certificates -  command: > -    /usr/bin/openshift admin create-server-cert -    --overwrite=false -    --cert={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/server.crt -    --key={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/server.key -    --hostnames={{ [item.openshift.common.hostname, -                    item.openshift.common.public_hostname]|unique|join(",") }} -  args: -    chdir: "{{ openshift_cert_dir_parent }}" -    creates: "{{ openshift_cert_dir_abs }}/node-{{ item.openshift.common.hostname }}/server.crt" -  with_items: openshift_nodes -  register: server_cert_result - -# TODO: create a failed_when condition -- name: Create node client certificates -  command: > -    /usr/bin/openshift admin create-node-cert -    --overwrite=false -    --cert={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/cert.crt -    --key={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/key.key -    --node-name={{ item.openshift.common.hostname }} -  args: -    chdir: "{{ openshift_cert_dir_parent }}" -    creates: "{{ openshift_cert_dir_abs }}/node-{{ item.openshift.common.hostname }}/cert.crt" -  with_items: openshift_nodes -  register: node_cert_result +# TODO: use a template lookup here  # TODO: create a failed_when condition -- name: Create kubeconfigs for nodes +- name: Use enterprise default for openshift_registry_url if not set +  set_fact: +    openshift_registry_url: "openshift3_beta/ose-${component}:${version}" +  when: openshift.common.deployment_type == 'enterprise' and openshift_registry_url is not defined +- name: Create node config    command: > -    /usr/bin/openshift admin create-kubeconfig -    --client-certificate={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/cert.crt -    --client-key={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/key.key -    --kubeconfig={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}/.kubeconfig -    --master={{ openshift.master.api_url }} -    --public-master={{ openshift.master.public_api_url }} +    /usr/bin/openshift admin create-node-config +      --node-dir={{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }} +      --node={{ item.openshift.common.hostname }} +      --hostnames={{ [item.openshift.common.hostname, item.openshift.common.public_hostname]|unique|join(",") }} +      --dns-domain={{ openshift.dns.domain }} +      --dns-ip={{ openshift.dns.ip }} +      --master={{ openshift.master.api_url }} +      --signer-key={{ openshift_master_ca_key }} +      --signer-cert={{ openshift_master_ca_cert }} +      --certificate-authority={{ openshift_master_ca_cert }} +      --signer-serial={{ openshift_master_ca_dir }}/serial.txt +      --node-client-certificate-authority={{ openshift_master_ca_cert }} +      {{ ('--images=' ~ openshift_registry_url) if openshift_registry_url is defined else '' }} +      --listen=https://0.0.0.0:10250    args: -    chdir: "{{ openshift_cert_dir_parent }}" -    creates: "{{ openshift_cert_dir_abs }}/node-{{ item.openshift.common.hostname }}/.kubeconfig" +    chdir: "{{ openshift_cert_parent_dir }}" +    creates: "{{ openshift_cert_dir }}/node-{{ item.openshift.common.hostname }}"    with_items: openshift_nodes -  register: kubeconfig_result  - name: Register unregistered nodes    kubernetes_register_node: -    client_user: openshift-client +    kubectl_cmd: ['openshift', 'kube']      name: "{{ item.openshift.common.hostname }}"      api_version: "{{ openshift_kube_api_version }}"      cpu: "{{ item.openshift.node.resources_cpu | default(None) }}" @@ -61,7 +45,5 @@      external_id: "{{ item.openshift.node.external_id }}"      # TODO: support customizing other attributes such as: client_config,      # client_cluster, client_context, client_user -    # TODO: update for v1beta3 changes after rebase: hostnames, external_ips, -    # internal_ips, external_id    with_items: openshift_nodes    register: register_result diff --git a/roles/openshift_register_nodes/vars/main.yml b/roles/openshift_register_nodes/vars/main.yml new file mode 100644 index 000000000..bd497f08f --- /dev/null +++ b/roles/openshift_register_nodes/vars/main.yml @@ -0,0 +1,7 @@ +--- +openshift_cert_parent_dir: /var/lib/openshift +openshift_cert_relative_dir: openshift.local.certificates +openshift_cert_dir: "{{ openshift_cert_parent_dir }}/{{ openshift_cert_relative_dir }}" +openshift_master_ca_dir: "{{ openshift_cert_dir }}/ca" +openshift_master_ca_cert: "{{ openshift_master_ca_dir }}/cert.crt" +openshift_master_ca_key: "{{ openshift_master_ca_dir }}/key.key" diff --git a/roles/openshift_repos/README.md b/roles/openshift_repos/README.md index 6713e11fc..6bbedd839 100644 --- a/roles/openshift_repos/README.md +++ b/roles/openshift_repos/README.md @@ -14,7 +14,7 @@ Role Variables  | Name                          | Default value |                                              |  |-------------------------------|---------------|----------------------------------------------| -| openshift_deployment_type     | online        | Possible values enterprise, origin, online   | +| openshift_deployment_type     | None          | Possible values enterprise, origin, online   |  | openshift_additional_repos    | {}            | TODO                                         |  Dependencies diff --git a/roles/openshift_repos/defaults/main.yaml b/roles/openshift_repos/defaults/main.yaml index 1730207f4..7c5a14cd7 100644 --- a/roles/openshift_repos/defaults/main.yaml +++ b/roles/openshift_repos/defaults/main.yaml @@ -1,7 +1,2 @@  --- -# TODO: once we are able to configure/deploy origin using the openshift roles, -# then we should default to origin - -# TODO: push the defaulting of these values to the openshift_facts module -openshift_deployment_type: online  openshift_additional_repos: {} diff --git a/roles/openshift_repos/files/online/epel7-kubernetes.repo b/roles/openshift_repos/files/online/epel7-kubernetes.repo deleted file mode 100644 index 1deae2939..000000000 --- a/roles/openshift_repos/files/online/epel7-kubernetes.repo +++ /dev/null @@ -1,6 +0,0 @@ -[maxamillion-epel7-kubernetes] -name=Copr repo for epel7-kubernetes owned by maxamillion -baseurl=http://copr-be.cloud.fedoraproject.org/results/maxamillion/epel7-kubernetes/epel-7-$basearch/ -skip_if_unavailable=True -gpgcheck=0 -enabled=1 diff --git a/roles/openshift_repos/files/online/RPM-GPG-KEY-redhat-beta b/roles/openshift_repos/files/online/gpg_keys/RPM-GPG-KEY-redhat-beta index 7b40671a4..7b40671a4 100644 --- a/roles/openshift_repos/files/online/RPM-GPG-KEY-redhat-beta +++ b/roles/openshift_repos/files/online/gpg_keys/RPM-GPG-KEY-redhat-beta diff --git a/roles/openshift_repos/files/online/RPM-GPG-KEY-redhat-release b/roles/openshift_repos/files/online/gpg_keys/RPM-GPG-KEY-redhat-release index 0f83b622d..0f83b622d 100644 --- a/roles/openshift_repos/files/online/RPM-GPG-KEY-redhat-release +++ b/roles/openshift_repos/files/online/gpg_keys/RPM-GPG-KEY-redhat-release diff --git a/roles/openshift_repos/files/online/epel7-openshift.repo b/roles/openshift_repos/files/online/repos/epel7-openshift.repo index c7629872d..c7629872d 100644 --- a/roles/openshift_repos/files/online/epel7-openshift.repo +++ b/roles/openshift_repos/files/online/repos/epel7-openshift.repo diff --git a/roles/openshift_repos/files/online/oso-rhui-rhel-7-extras.repo b/roles/openshift_repos/files/online/repos/oso-rhui-rhel-7-extras.repo index cfe41f691..cfe41f691 100644 --- a/roles/openshift_repos/files/online/oso-rhui-rhel-7-extras.repo +++ b/roles/openshift_repos/files/online/repos/oso-rhui-rhel-7-extras.repo diff --git a/roles/openshift_repos/files/online/oso-rhui-rhel-7-server.repo b/roles/openshift_repos/files/online/repos/oso-rhui-rhel-7-server.repo index ddc93193d..ddc93193d 100644 --- a/roles/openshift_repos/files/online/oso-rhui-rhel-7-server.repo +++ b/roles/openshift_repos/files/online/repos/oso-rhui-rhel-7-server.repo diff --git a/roles/openshift_repos/files/online/rhel-7-libra-candidate.repo b/roles/openshift_repos/files/online/repos/rhel-7-libra-candidate.repo index b4215679f..b4215679f 100644 --- a/roles/openshift_repos/files/online/rhel-7-libra-candidate.repo +++ b/roles/openshift_repos/files/online/repos/rhel-7-libra-candidate.repo diff --git a/roles/openshift_repos/files/origin/repos/maxamillion-origin-next-epel-7.repo b/roles/openshift_repos/files/origin/repos/maxamillion-origin-next-epel-7.repo new file mode 100644 index 000000000..0b21e0a65 --- /dev/null +++ b/roles/openshift_repos/files/origin/repos/maxamillion-origin-next-epel-7.repo @@ -0,0 +1,7 @@ +[maxamillion-origin-next] +name=Copr repo for origin-next owned by maxamillion +baseurl=https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/epel-7-$basearch/ +skip_if_unavailable=True +gpgcheck=1 +gpgkey=https://copr-be.cloud.fedoraproject.org/results/maxamillion/origin-next/pubkey.gpg +enabled=1 diff --git a/roles/openshift_repos/tasks/main.yaml b/roles/openshift_repos/tasks/main.yaml index bb1551d37..12e98b7a1 100644 --- a/roles/openshift_repos/tasks/main.yaml +++ b/roles/openshift_repos/tasks/main.yaml @@ -10,10 +10,6 @@  - assert:      that: openshift_deployment_type in known_openshift_deployment_types -# TODO: remove this when origin support actually works -- fail: msg="OpenShift Origin support is not currently enabled" -  when: openshift_deployment_type == 'origin' -  - name: Ensure libselinux-python is installed    yum:      pkg: libselinux-python @@ -36,17 +32,15 @@      path: "/etc/yum.repos.d/{{ item | basename }}"      state: absent    with_fileglob: -  - '*/*' -  when: not (item | search("/files/" + openshift_deployment_type + "/")) and (item | search(".repo$")) +  - '*/repos/*' +  when: not (item | search("/files/" ~ openshift_deployment_type ~ "/repos"))  - name: Configure gpg keys if needed    copy: src={{ item }} dest=/etc/pki/rpm-gpg/    with_fileglob: -  - "{{ openshift_deployment_type }}/*" -  when: item | basename | match("RPM-GPG-KEY-") +  - "{{ openshift_deployment_type }}/gpg_keys/*"  - name: Configure yum repositories    copy: src={{ item }} dest=/etc/yum.repos.d/    with_fileglob: -  - "{{ openshift_deployment_type }}/*" -  when: item | basename | search(".*\.repo$") +  - "{{ openshift_deployment_type }}/repos/*" diff --git a/roles/openshift_repos/templates/yum_repo.j2 b/roles/openshift_repos/templates/yum_repo.j2 index 7ea2c7460..2d9243545 100644 --- a/roles/openshift_repos/templates/yum_repo.j2 +++ b/roles/openshift_repos/templates/yum_repo.j2 @@ -1,4 +1,3 @@ -# {{ ansible_managed }}  {% for repo in openshift_additional_repos %}  [{{ repo.id }}]  name={{ repo.name | default(repo.id) }} diff --git a/roles/openshift_sdn_master/tasks/main.yml b/roles/openshift_sdn_master/tasks/main.yml index f2d61043b..77e7a80ba 100644 --- a/roles/openshift_sdn_master/tasks/main.yml +++ b/roles/openshift_sdn_master/tasks/main.yml @@ -12,12 +12,21 @@    yum:      pkg: openshift-sdn-master      state: installed +  register: install_result +- name: Reload systemd units +  command: systemctl daemon-reload +  when: install_result | changed + +# TODO: we should probably generate certs specifically for sdn  - name: Configure openshift-sdn-master settings    lineinfile:      dest: /etc/sysconfig/openshift-sdn-master      regexp: '^OPTIONS=' -    line: "OPTIONS=\"-v={{ openshift.master_sdn.debug_level }}\"" +    line: "OPTIONS=\"-v={{ openshift.master_sdn.debug_level }} -etcd-endpoints={{ openshift_sdn_master_url}} +      -etcd-cafile={{ openshift_cert_dir }}/ca/ca.crt +      -etcd-certfile={{ openshift_cert_dir }}/openshift-client/cert.crt +      -etcd-keyfile={{ openshift_cert_dir }}/openshift-client/key.key\""    notify:    - restart openshift-sdn-master diff --git a/roles/openshift_sdn_node/tasks/main.yml b/roles/openshift_sdn_node/tasks/main.yml index 729c28879..c2329dd6f 100644 --- a/roles/openshift_sdn_node/tasks/main.yml +++ b/roles/openshift_sdn_node/tasks/main.yml @@ -9,9 +9,15 @@    yum:      pkg: openshift-sdn-node      state: installed +  register: install_result + +- name: Reload systemd units +  command: systemctl daemon-reload +  when: install_result | changed  # TODO: we are specifying -hostname= for OPTIONS as a workaround for  # openshift-sdn-node not properly detecting the hostname. +# TODO: we should probably generate certs specifically for sdn  - name: Configure openshift-sdn-node settings    lineinfile:      dest: /etc/sysconfig/openshift-sdn-node @@ -20,7 +26,10 @@      backrefs: yes    with_items:      - regex: '^(OPTIONS=)' -      line: '\1"-v={{ openshift.node_sdn.debug_level }} -hostname={{ openshift.common.hostname }}"' +      line: '\1"-v={{ openshift.node_sdn.debug_level }} -hostname={{ openshift.common.hostname }} +        -etcd-cafile={{ openshift_node_cert_dir }}/ca.crt +        -etcd-certfile={{ openshift_node_cert_dir }}/client.crt +        -etcd-keyfile={{ openshift_node_cert_dir }}/client.key\"'      - regex: '^(MASTER_URL=)'        line: '\1"{{ openshift_sdn_master_url }}"'      - regex: '^(MINION_IP=)' diff --git a/roles/os_firewall/library/os_firewall_manage_iptables.py b/roles/os_firewall/library/os_firewall_manage_iptables.py index 90588d2ae..9d0af497d 100755 --- a/roles/os_firewall/library/os_firewall_manage_iptables.py +++ b/roles/os_firewall/library/os_firewall_manage_iptables.py @@ -270,4 +270,5 @@ def main():  # import module snippets  from ansible.module_utils.basic import * -main() +if __name__ == '__main__': +    main() | 
