Brevet/dev

Development

Ubuntu Host Requirements

Check kvm & virtualbox

sudo kvm-ok
sudo apt install -y qemu qemu-kvm libvirt-daemon libvirt-clients bridge-utils virt-manager

sudo apt install virtualbox

Or install Virtualbox from https://www.virtualbox.org/wiki/Downloads

Install vagrant & ansible

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install vagrant

vagrant plugin install vagrant-vbguest

vagrant plugin install vagrant-hostmanager

vagrant plugin install vagrant-disksize

ansible-galaxy collection install ansible.posix

ansible-galaxy collection install community.crypto

ansible-galaxy collection install community.general

ansible-galaxy collection install community.mysql

ansible-galaxy collection install community.aws

In case of using ceph, do:

export VAGRANT_EXPERIMENTAL="disks"
vagrant reload

Deploy

vagrant up

• In case of "'/var/run/libvirt/libvirt-sock': Permission denied", try to login/logout or restart the machine

• In case of Rocky/8 and "Stderr: 0%...VBOX_E_OBJECT_NOT_FOUND", try the following bugfix box metadata file. Open a new file called box-metadata.json and write:

{
  "name" : "rockylinux/8",
  "description" : "Rocky Linux 8 7.0.0 Bugfix",
  "versions" : [
    {
      "version" : "7.0.1-20221213.0",
      "providers" : [
        {
          "name" : "virtualbox",
          "url" : "http://dl.rockylinux.org/pub/rocky/8/images/x86_64/Rocky-8-Vagrant-Vbox-8.7-20221213.0.x86_64.box"
        }
      ]
    }
  ]
}

Now apply the patch with:

vagrant box add box-metadata.json

• In case of you need disk extension (eg. conf.disksize.size = '100GB' with plugin vagrant-disksize), you must resize the filesystem from within the guest:

vagrant ssh guest
sudo su
parted -l
fdisk -l /dev/sda
fdisk /dev/sda
fdisk -l /dev/sda
mkfs -t xfs -f /dev/sda5
mkdir /extent
mount -t xfs -o defaults /dev/sda5 /extent
blkid
vi /etc/fstab
exit
exit
vagrant reload guest

Apply Ansible

Once all VMs are up (see 'Deploy'), you can lauch Ansible without Vagrant. Examples:

ansible -v -i '192.168.56.41,' --key-file .vagrant/machines/lb/virtualbox/private_key -u vagrant -b -m setup all

ansible-playbook -v -i provisioning/ansible/hosts -u vagrant -b provisioning/ansible/playbook.yml

Result

In your /etc/hosts file, add a line to match the dev nextcloud domain (defined in your Vagrantfile), eg "nextcloud.test", to the choosen IP, eg "192.168.56.51".

Open a browser with "https://nextcloud.test"

Validation

1 DB - 1 Web - 1 LB

Done with:

• Apache & mod_php
• no Redis

HOSTS = [ 
  { :hostname => "db1",         :ip => NETWORK+"11",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers"   }, 
  { :hostname => "web.test",    :ip => NETWORK+"41",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "web_servers"  },
  { :hostname => "lb.test",     :ip => NETWORK+"51",  :ram => 1024,  :cpu => 1,  :box => "ubuntu/focal64", :group => "lbal_servers" },
]

3 DBs galera cluster - 1 Web - 1 LB

OK

TODO: install KeepAlived in DB nodes

HOSTS = [ 
  { :hostname => "db1",         :ip => NETWORK+"11",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers"   },
  { :hostname => "db2",         :ip => NETWORK+"12",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers"   },
  { :hostname => "db3",         :ip => NETWORK+"13",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers"   },
  { :hostname => "web.test",    :ip => NETWORK+"41",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "web_servers"  },
  { :hostname => "lb.test",     :ip => NETWORK+"51",  :ram => 1024,  :cpu => 1,  :box => "ubuntu/focal64", :group => "lbal_servers" },
]

3 DBs galera cluster - 2 ProxySQL - 1 Web - 1 LB

OK

HOSTS = [
  { :hostname => "db1",         :ip => NETWORK+"11",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers",      },
  { :hostname => "db2",         :ip => NETWORK+"12",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers",      },
  { :hostname => "db3",         :ip => NETWORK+"13",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers",      },
  { :hostname => "lbsql1",      :ip => NETWORK+"19",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_lbal_servers", :state => "MASTER",  :priority => 101, :vip => NETWORK+"20" },
  { :hostname => "lbsql2",      :ip => NETWORK+"18",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_lbal_servers", :state => "BACKUP",  :priority => 100, :vip => NETWORK+"20" },
  { :hostname => "web.test",    :ip => NETWORK+"41",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "web_servers",     :ipdb => NETWORK+"20" },
  { :hostname => "lb.test",     :ip => NETWORK+"51",  :ram => 1024,  :cpu => 1,  :box => "ubuntu/focal64", :group => "lbal_servers"     },
]

1 DB - 2 GL - 2 Web - 2 LB

OK

Done with:

• Apache & mod_php
• KeyDB instead of Redis
• KeepAlived & HAProxy for redis

HOSTS = [
  { :hostname => "db1",         :ip => NETWORK+"11",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "db_servers",      }, 
  { :hostname => "gl1",         :ip => NETWORK+"31",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "gluster_servers"  },
  { :hostname => "gl2",         :ip => NETWORK+"32",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "gluster_servers"  },
  { :hostname => "web.test",    :ip => NETWORK+"41",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "web_servers",     :redisd => "keydb", :redisp => "6380", :redisv => NETWORK+"40", :priority => 101 },
  { :hostname => "web2.test",   :ip => NETWORK+"42",  :ram => 1024,  :cpu => 1,  :box => "centos/7",       :group => "web_servers",     :redisd => "keydb", :redisp => "6380", :redisv => NETWORK+"40", :priority => 100 },
  { :hostname => "lb.test",     :ip => NETWORK+"51",  :ram => 1024,  :cpu => 1,  :box => "ubuntu/focal64", :group => "lbal_servers",    :state => "MASTER",  :priority => 101 },
  { :hostname => "lb2.test",    :ip => NETWORK+"52",  :ram =>  512,  :cpu => 1,  :box => "ubuntu/focal64", :group => "lbal_servers",    :state => "BACKUP",  :priority => 100 },
]

3 DB galera cluster - 2 ProxySQL - 2 GL - 2 Web - 2 LB

TODO

• Nginx instead of Apache
• Object storage
• Rocky 8 instead of Centos 7
• Prometheus & Grafana investigation
• Dev : Vagrant ==> OpenStack + Ceph
• Test : OpenStack + Ceph => benchmarks
• Prod

HOSTS = [
]

Apply Ansible on Openstack

Examples:

ansible -v -i '192.168.64.68,' --key-file /home/nextcloud/Documents/Secure/Unix/ssh/id_rsa_pedro -u pedro -b -m setup all

ansible-playbook -v -i provisioning/ansible/hosts_openstack -b provisioning/ansible/playbook_openstack.yml