Voici comment installer un serveur Puppet V4 (open source) complet comprenant:
- Puppetserver
- Puppetdb + Postgres
- Activemq + Mcollective + plugins Mcollective
- Hiera
- R10k
- Puppetboard
- Puppetexplorer
- Puppetdbquery
- Modules usuels
- Outils usuels
C’est l’équivalent d’un serveur AIO (All In One) de Puppet Enterprise.
A noter que l’on peut aussi se servir de ce tutoriel pour installer l’équivalent d’un serveur Split de Puppet Enterprise en répartissant les différentes briques ci-dessus sur plusieurs serveurs. Dans ce cas la répartition habituelle sera de ce type:
- Serveur MOM (Master Of Master)
- Puppetserver
- CA
- Activemq + serveur Mcollective
- Hiera
- R10k
- Serveur Puppetdb
- Puppetdb
- Postgres
- Client Mcollective
- Serveur Console
- Puppetboard
- Client Mcollective
- Serveur Master compilation
- Puppetserver
- Client Mcollective
- Hiera
- R10k
On peut installer plusieurs serveurs de compilation selon le nombre d’agent Puppet à gérer (horizontal scaling), même chose concernant puppetdb, pour postgres il faut passer en mode cluster et réplication.
Attention de bien spécifier tous les serveurs de compilation dans la directive « dns_alt_names » du puppet.conf du serveur MOM dans ce cas.
Prérequis: Disposer d’un serveur Linux de type RHEL/CENTOS 6.x avec au minimum 2 Go de RAM
Partie 1 : Puppetserver + CA
yum -y install https://yum.puppetlabs.com/puppetlabs-release-pc1-el-6.noarch.rpm yum -y install puppet-agent puppetserver cat > /etc/puppetlabs/puppet/puppet.conf << EOF [master] vardir = /opt/puppetlabs/server/data/puppetserver logdir = /var/log/puppetlabs/puppetserver rundir = /var/run/puppetlabs/puppetserver pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid codedir = /etc/puppetlabs/code dns_alt_names = $(hostname),$(hostname -f),puppet [main] certname = $(hostname -f) server = $(hostname -f) EOF chkconfig puppet on chkconfig puppetserver on /opt/puppetlabs/bin/puppet master --no-daemonize —verbose Crtl ^C
Partie 2 : Puppetdb + Postgres
yum -y install puppetdb puppetdb-termini /opt/puppetlabs/bin/puppetdb ssl-setup chkconfig puppetdb on cat > /etc/puppetlabs/puppet/routes.yaml << EOF --- master: facts: terminus: puppetdb cache: yaml EOF cat > /etc/puppetlabs/puppet/puppetdb.conf << EOF [main] server_urls = https://$(hostname -f):8081 EOF yum -y install https://yum.postgresql.org/9.5/redhat/rhel-6-x86_64/pgdg-centos95-9.5-3.noarch.rpm yum -y install postgresql95-server postgresql95 postgresql95-libs postgresql95-contrib puppetdb-terminus chkconfig postgresql-9.5 on service postgresql-9.5 initdb cat > /var/lib/pgsql/9.5/data/pg_hba.conf << EOF # TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all peer # IPv4 local connections: host all all 127.0.0.1/32 md5 # IPv6 local connections: host all all ::1/128 md5 EOF service postgresql-9.5 start su - postgres createuser -DRSP puppetdb createdb -O puppetdb puppetdb psql \c puppetdb CREATE EXTENSION pg_trgm; exit cat > /etc/puppetlabs/puppetdb/conf.d/database.ini << EOF [database] classname = org.postgresql.Driver subprotocol = postgresql subname = //127.0.0.1:5432/puppetdb username = puppetdb password = puppetdb log-slow-statements = 10 EOF cat > /etc/puppetlabs/puppetdb/conf.d/jetty.ini << EOF [jetty] # IP address or hostname to listen for clear-text HTTP. To avoid resolution # issues, IP addresses are recommended over hostnames. # Default is localhost. host = $(hostname -f) # Port to listen on for clear-text HTTP. port = 8080 # The following are SSL specific settings. They can be configured # automatically with the tool puppetdb ssl-setup, which is normally # ran during package installation. # IP address to listen on for HTTPS connections. Hostnames can also be used # but are not recommended to avoid DNS resolution issues. To listen on all # interfaces, use 0.0.0.0. ssl-host = 0.0.0.0 # The port to listen on for HTTPS connections ssl-port = 8081 # Private key path ssl-key = /etc/puppetlabs/puppetdb/ssl/private.pem # Public certificate path ssl-cert = /etc/puppetlabs/puppetdb/ssl/public.pem # Certificate authority path ssl-ca-cert = /etc/puppetlabs/puppetdb/ssl/ca.pem # Access logging configuration path. To turn off access logging # comment out the line with access-log-config=... access-log-config = /etc/puppetlabs/puppetdb/request-logging.xml EOF cat > /etc/puppetlabs/puppetdb/conf.d/config.ini << EOF # See README.md for more thorough explanations of each section and # option. [global] # Store mq/db data in a custom directory vardir = /opt/puppetlabs/server/data/puppetdb # Use an external logback config file logging-config = /etc/puppetlabs/puppetdb/logback.xml [command-processing] # How many command-processing threads to use, defaults to (CPUs / 2) # threads = 4 # Maximum amount of disk space (in MB) to allow for ActiveMQ persistent message storage # store-usage = 102400 # Maximum amount of disk space (in MB) to allow for ActiveMQ temporary message storage # temp-usage = 51200 EOF cat > /etc/puppetlabs/puppet/puppet.conf << EOF [master] vardir = /opt/puppetlabs/server/data/puppetserver logdir = /var/log/puppetlabs/puppetserver rundir = /var/run/puppetlabs/puppetserver pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid codedir = /etc/puppetlabs/code dns_alt_names = $(hostname),$(hostname -f),puppet [main] certname = $(hostname -f) server = $(hostname -f) pluginsync = true storeconfigs = true storeconfigs_backend = puppetdb reports = store,puppetdb EOF chown -R puppet:puppet /etc/puppetlabs chown -R puppetdb:puppetdb /etc/puppetlabs/puppetdb service puppetdb restart service puppetserver restart
Partie 3 : Activemq + Mcollective
yum -y install http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm yum -y install activemq *mcollective* cat /etc/activemq/activemq.xml << EOF <beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd"> <broker xmlns="http://activemq.apache.org/schema/core" brokerName="$(hostname -f)" useJmx="true"> <managementContext> <managementContext createConnector="false"/> </managementContext> <plugins> <statisticsBrokerPlugin/> <simpleAuthenticationPlugin> <users> <authenticationUser username="mcollective" password="marionette" groups="mcollective,everyone"/> <authenticationUser username="admin" password="admin" groups="mcollective,admins,everyone"/> </users> </simpleAuthenticationPlugin> <authorizationPlugin> <map> <authorizationMap> <authorizationEntries> <authorizationEntry queue=">" write="admins" read="admins" admin="admins" /> <authorizationEntry topic=">" write="admins" read="admins" admin="admins" /> <authorizationEntry topic="mcollective.>" write="mcollective" read="mcollective" admin="mcollective" /> <authorizationEntry queue="mcollective.>" write="mcollective" read="mcollective" admin="mcollective" /> <authorizationEntry topic="ActiveMQ.Advisory.>" read="everyone" write="everyone" admin="everyone"/> </authorizationEntries> </authorizationMap> </map> </authorizationPlugin> </plugins> <systemUsage> <systemUsage> <memoryUsage> <memoryUsage limit="20 mb"/> </memoryUsage> <storeUsage> <storeUsage limit="1 gb" name="foo"/> </storeUsage> <tempUsage> <tempUsage limit="100 mb"/> </tempUsage> </systemUsage> </systemUsage> <transportConnectors> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/> <transportConnector name="stomp+nio" uri="stomp://0.0.0.0:61613"/> </transportConnectors> </broker> </beans> EOF ln -s /usr/share/activemq/data /usr/share/activemq/activemq-data chkconfig activemq on service activemq start cat > /etc/puppetlabs/mcollective/server.cfg << EOF main_collective = mcollective collectives = mcollective libdir = /opt/puppetlabs/puppet/lib/ruby/vendor_ruby:/opt/puppetlabs/mcollective/plugins:/usr/share/mcollective/plugins:/usr/libexec/mcollective logfile = /var/log/puppetlabs/mcollective.log loglevel = info daemonize = 1 direct_addressing = 0 # Plugins securityprovider = psk plugin.psk = unset connector = activemq plugin.activemq.pool.size = 1 plugin.activemq.pool.1.host = $(hostname -f) plugin.activemq.pool.1.port = 61613 plugin.activemq.pool.1.user = mcollective plugin.activemq.pool.1.password = marionette plugin.package.provider = puppet # Facts factsource = yaml plugin.yaml = /etc/puppetlabs/mcollective/facts.yaml # Registration registerinterval = 300 registration = Agentlist EOF chkconfig mcollective on chown -R puppet:puppet /etc/puppetlabs/mcollective /opt/puppetlabs/bin/gem install net-ping ln -s /usr/libexec/mcollective/mcollective /opt/puppetlabs/mcollective/plugins/ echo "0,15,30,45 * * * * /opt/puppetlabs/bin/facter -py > /etc/puppetlabs/mcollective/facts.yaml" >> /var/spool/cron/root service crond reload service mcollective start cat > ~/.mcollective << EOF main_collective = mcollective collectives = mcollective libdir = /opt/puppetlabs/puppet/lib/ruby/vendor_ruby:/opt/puppetlabs/mcollective/plugins logfile = /dev/null loglevel = info direct_addressing = 0 # Plugins securityprovider = psk plugin.psk = unset plugin.package.provider = puppet connector = activemq plugin.activemq.pool.size = 1 plugin.activemq.pool.1.host = $(hostname -f) plugin.activemq.pool.1.port = 61613 plugin.activemq.pool.1.user = mcollective plugin.activemq.pool.1.password = marionette plugin.activemq.pool.1.ssl = false # Facts factsource = yaml plugin.yaml = /etc/puppetlabs/mcollective/facts.yaml default_discovery_method = mc EOF mco ping
Partie 4 : Hiera
ln -s /etc/puppetlabs/puppet/hiera.yaml /etc cat > /etc/hiera.yaml << EOF --- :backends: - yaml :hierarchy: - "hieradata/%{hostname}/%{module_name}" - "hieradata/%{hostname}/common" - "environments%{environment}/modules/%{module_name}/defaults" :yaml: :datadir: /etc/puppetlabs/code/ EOF puppetserver gem install hiera chown -R puppet:puppet /etc/puppetlabs/puppet
Partie 5 : Modules usuels + liens symboliques vers les outils Puppet
# Autre solution pour l'installation de ces modules utiliser r10k (voir partie 6 et 7) puppet module install puppetlabs/stdlib puppet module install puppetlabs/concat ln -s /opt/puppetlabs/puppet/bin/gem /opt/puppetlabs/bin/gem ln -s /opt/puppetlabs/puppet/bin/augparse /opt/puppetlabs/bin/augparse ln -s /opt/puppetlabs/puppet/bin/augtool /opt/puppetlabs/bin/augtool
Partie 6 : serveur git local pour r10k et définition des controls repos
adduser git yum -y install git mkdir .ssh chmod 700 .ssh touch .ssh/authorized_keys chmod 600 .ssh/authorized_keys ssh-keygen cat ~/.ssh/id_rsa.pub > ~git/.ssh/authorized_keys su - git for i in environments.git hieradata.git manifests.git do mkdir $i cd $i git init --bare done exit for i in environments hieradata manifests do mkdir -p git/$i done cd environments git init cat > Puppetfile << EOF mod 'puppetlabs/stdlib', :latest mod 'puppetlabs/concat', :latest mod 'apache', :local => true mod 'epel', :local => true mod 'puppetexplorer', :local => true mod 'puppetboard', :local => true mod 'python', :local => true mod 'vcsrepo', :local => true EOF cat > environment.conf << EOF modulepath = /etc/puppetlabs/code/environments/production/modules:$basemodulepath manifest = /etc/puppetlabs/code/manifests EOF git add Puppetfile environment.conf git commit -m 'initial commit' git branch -m master production git remote add origin git@git:~/environments.git git push origin production cd ../hieradata git init mkdir $(hostname) echo "---" > $(hostname)/common.yaml cat > $(hostname)/puppetexplorer.yaml << EOF puppetexplorer::vhost_options: rewrites: - rewrite_rule: - '^/api/metrics/v1/mbeans/puppetlabs.puppetdb.query.population:type=default,name=(.*)$ https://%%{}{HTTP_HOST}/api/metrics/v1/mbeans/puppetlabs.puppetdb.population:name=$1 [R=301,L]' EOF git add $(hostname)/common.yaml $(hostname)/puppetexplorer.yaml git commit -m 'initial commit' git branch -m master hieradata git remote add origin git@git:~/hieradata.git git push origin hieradata cd ../manifests git init cat > init.pp << EOF ## site.pp ## File { backup => false } EOF cat > $(hostname).pp << EOF # Configure Apache class { 'apache': purge_configs => false, mpm_module => 'prefork', default_vhost => true, default_mods => false, } class { 'apache::mod::wsgi': } class { 'apache::mod::proxy': } class { 'apache::mod::dir': } # Configure Puppetboard class { 'puppetboard': manage_git => 'latest', manage_virtualenv => 'latest', enable_catalog => true, experimental => true, } # Configure Puppetexplorer include puppetexplorer EOF git add $(hostname).pp init.pp git commit -m 'initial commit' git branch -m master manifests git remote add origin git@git:~/manifests.git git push origin manifests
Partie 7 : r10k
gem install r10k ln -s /opt/puppetlabs/puppet/bin/r10k /opt/puppetlabs/bin/ mkdir /etc/puppetlabs/r10k cat > /etc/puppetlabs/r10k/r10k.yaml << EOF cachedir: '/var/cache/r10k' sources: environments: remote: 'git@git:~/environments.git' basedir: '/etc/puppetlabs/code/environments' manifests: remote: 'git@git:~/manifests.git' basedir: '/etc/puppetlabs/code' hieradata: remote: 'git@git:~/hieradata.git' basedir: '/etc/puppetlabs/code' git: provider: shellgit postrun: ['/bin/chown', '-R', 'puppet:puppet', '/etc/puppetlabs/code' EOF chown -R puppet:puppet /etc/puppetlabs/r10k r10k deploy environment -p
Partie 8 : Puppetboard (la configuration du module est visible en partie 6)
puppet module install puppet-puppetboard puppet module install puppetlabs-apache puppet agent -t
Puppetboard url : http://@ip/puppetboard
Partie 9 : Puppetexplorer (la configuration du module est visible en partie 6)
puppet module install spotify/puppetexplorer puppet agent -t
Puppetexplore url: https://@ip/
Partie 10 : Puppetdbquery
wget https://raw.githubusercontent.com/f951753/puppetdbquery/master/puppetdbquery > /usr/local/bin/puppetdbquery wget https://raw.githubusercontent.com/f951753/puppetdbquery/master/addreadonly2database > /usr/local/bin/addreadonly2database chmod 755 /usr/local/bin/puppetdbquery /usr/local/bin/addreadonly2database su - postgres addreadonly2database exit puppetdbquery
Partie 11 : Outils usuels
gem install puppet-lint gem install metadata-json-lint gem install puppet-check gem install r10kdiff
Un fichier « ova » est disponible ICI pour ce serveur AIO Puppet v4 tel que décrit ci-dessus.
NB: Attention actuellement, la dernière version de la librairie python Werkzeug (v0.12.1) casse Puppetboard, il faut revenir à la version v0.11.15 pour voir Puppetboard fonctionner… https://github.com/voxpupuli/puppetboard/issues/370 pour plus d’info.