DIY DevOps
DIY DevOps
About the Book
This tutorial is for anyone or team who needs to automate the deployment and maintenance of static websites (html/css/javascript) on a Virtual Private Server instance at digitalocean.com. If you are capable of following tutorials to create a very basic HTML file, you should have no problem following this tutorial.
We will set up websites on a digitalocean.com's droplet (aka Virtual Private Server).
When I started self-hosting websites on VPS, I realized there was a lot of repetitive work involved in deploying content. Even fixing a small typo involved way too many manual steps. To fix a simple typo, I had to fix the typo on local computer > FTP or scp files up > ssh into web server > copy uploadeded .html file into place > reapply permission. Simply too many manual steps for fixing a typo. I wanted to automate the process so that all of the steps can be done with least amount of manual work. As I was teaching myself how to automate it, I had to turn to many different tutorials/videos/documentations. Unfortunately many tutorials did not exactly fit into my scenarios. I slowly taught myself how to setup/run/automate self-hosted website and slowly became a self-taught DIY DevOps engineer. Obviously a DevOps engineer has many roles, but one important role is automating website content deployment.
Here is a short list of tasks we will complete in this tutorial.
- Buy a domain name at namecheap.com.
- Create a SSH key pair on local macOS.
- Upload SSH public key to digitalocean.com.
- Start a new VPS at digitalocean. We will nickname the new VPS as dserver01.
- Rebuild the new VPS, dserver01.
- Install Apache webserver.
- Sign up with cloudflare.com.
- Transfer name server from namecheap.com to cloudflare.com.
- Configure dserver01 with a shell script.
- Disable ssh login by root user.
- Set up 3 websites on Apache.
- Install nginx.
- Configure nginx as a reverse proxy, sitting between internet and Apache web server.
- Install LetsEncrypt.
- Set up .well-known in nginx.
- Get SSL certificates
- Set up nginx to use SSL certificates. Our webserver now serves content encrypted by SSL.
- Configure Apache and nginx to log with real IP of visitors.
- Deploy sample HTML5 website using a HTML/CSS/JS template.
- Using shell script to automate code deployment.
- Using Git
Following topics will be covered in future updates.
- Using Jenkins
- Script to automate configuring apache/nginx to add new websites
About the Author
Table of Contents
-
Chapter 1: Introduction
- Summary of tasks covered
- Tools used
- Low cost
- Website URLs
- 3 types of hosting platform
-
Chapter 2: Preparation
- CLI and SSH
- Terminal versus iTerm2
- Incremental changes
- Target audience
- cost for domain/VPS
-
Chapter 3: Buy domain name and VPS
- Get a domain name to play with - namecheap.com
- Check status of the domain name on icann.org
- Verify with ICANN
- Create a SSH key pair
- Digital Ocean
- Add ssh public key to Digital Ocean
- Start up a Digital Ocean droplet.
- Initial ssh into droplet: dserver01
- Danger with SSH and different computers involved
- Create a test file on dserver01
- Rebuild versus destroy a droplet
- Rebuild a droplet
- SSHing into a rebuilt droplet
- Log into rebuilt droplet (dserver01)
- Firewalld on droplet (dserver01)
- Browser choice
- Verify server IP on web page
- Firefox Private Window
- Test install Apache web server
- Status check
- Signing up with CloudFlare
- Transfer name server from namecheap.com to cloudflare.com
- Test browsing devopsdiy.xyz (use your domain)
- Cloudflare.com: change IP for A record of devopsdiy.xyz
- Cloudflare.com: CNAME for www.devopsdiy.xyz
- www versus non-www URL
- Cloudflare.com: add 2 more subdomains
- cloudflare.com and DNS changes
- CloudFlare: “Proxied” vs. “DNS only”
- SSH into dserver01 with hostname
- Status check
-
Chapter 4: Basic configuration of dserver01
- yum update
- Configure dserver01 server
- Quick primer on bash script
- First bash script
- cat, less
- bash script with functions
- bash script to set up CentOS 7
-
useraandtestuser1plus ssh public key -
Update SSH public key in
setup01-devopsdiy.sh - Self deleting setup01-devopsdiy.sh
- firewalld and setup01-devopsdiy.sh
- SSH into dserver01.
- Check if screen is installed
-
Run
setup01-devopsdiy.sh -
Test logging in as
userawith sudo privilege - failed login attempts
-
Verify
screenis installed -
Self-delete of
setup01-devopsdiy.sh -
Disable ssh login for
root
-
Chapter 5: Install Apache
- Update httpd.conf
- Configure Apache for devopsdiy.xyz
- Add 2 more domains for dev and qa work
-
Chapter 6: Install nginx
- nginx
- screen - virtual session
- tmux
- openssl dhparam
- Prepare Apache to work with nginx
- Free up port 80
- Install nginx
- Configure nginx
- Server blocks in nginx
- devel, qatest and www
- systemctl enable
- Status
-
Chapter 7: LetsEncrypt
- LetsEncrypt
- Install LetsEncrypt
- .well-known
- Get ssl cert for all 3 domains
- LetsEncrypt renewal test
- Update nginx config files to use SSL
- ssl config for devopsdiy.xyz
- ssl and www to non-www redirect
- Update devel.devopsdiy.xyz.proxy.conf file to use SSL
- Update qatest.devopsdiy.xyz.proxy.conf file to use SSL
- Get A+ instead of B from https://www.ssllabs.com/ssltest/
-
Chapter 8: Logging
- Logging source IP into Apache log files
- tail -f in multiple iTerm2 panes
- Real IP
- access.log: logs before update
- access.log: logs we want to see after update
- error.log: logs before update
- Updating Apache Global configuration
- nginx Global configuration update
- update Virtual Host configuration
- Testing logging with Proxy turned off and on
- Logrotate
- Install brew and curl
- Generate logs for test
- bin folder
- Saving command output into a file
- Tools to use
- Update logrotate for httpd
- Logrotate for low volume production
- Logrotate for high volume production
-
Chapter 9: deploy content with rsync/script
- Get HTML5 template website
- Prep on macOS to upload files
- Uploading website content to dserver01
- Check uploaded files on dserver01
- Get deploy-html-runonserver.sh
- Initial run of deploy-html-runonserver.sh
- Deploying to devel, qatest and production
- Deploy to devel site
- Deploy to QAtest site
- Try deploy to Production site without -f y
- Deploy to Production site with -f y
- uptimerobot.com
-
Chapter 10: Git repository
- Set up Git repo
- Create a new repository
- Add SSH Public key on github
- Git clone the git repo
- Git checkin - initial
- Add one file to git repo
- Deploying content directly from git repo to web server
-
Chapter 11: Jenkins
- Jenkins and Git
- Setting up Jenkins server on CentOS 7
- VPS for jenkserv01
- Install git
- Install Jenkins
- Install nginx
- Static website by nginx
- Install LetsEncrypt
- openssl dhparam in screen
- .well-known folder for nginx
- Create SSL cert
- Reverse Proxy for Jenkins
- Block port 8080
- Initial log in into Jenkins
- Updating Jenkins
-
SSH key pair for user
jenkinson jenkserv01 - Prep dserver01 for Jenkins jobs
- Test sshing in from jenkserv01 to dserver01
- Github and Deploy key (SSH public key)
- Create zero-out-site.sh
- Create monitor-folder-size.sh
- Prep iTerm2 window for use in test
- Create a Jenkins job to use with Git repo
- Add SSH Servers in Jenkins
- Create a Jenkins job to use with rsynced up files
- Available deployment methods
- Jenkins Global Security setup
- Prepare Jenkins job to run with curl
-
Verify Jenkins job
devopsdiy.xyz-rsyncfor curl - curl to run a Jenkins job
- Available deployment methods
- Issues with using curl deploy method
- Use deploy-html-jenkins.sh
- Testing deploy-html-jenkins.sh
- Using deploy-html-jenkins.sh
-
Source code
- add-user-jenkagent.sh
- apache-logrotate-test-tool.sh
- deploy-html-jenkins.sh
- deploy-html-runonserver.sh
- setup01-devopsdiy.sh
- zero-out-site.sh
The Leanpub 60 Day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
Now, this is technically risky for us, since you'll have the book or course files either way. But we're so confident in our products and services, and in our authors and readers, that we're happy to offer a full money back guarantee for everything we sell.
You can only find out how good something is by trying it, and because of our 100% money back guarantee there's literally no risk to do so!
So, there's no reason not to click the Add to Cart button, is there?
See full terms...
Earn $8 on a $10 Purchase, and $16 on a $20 Purchase
We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.
(Yes, some authors have already earned much more than that on Leanpub.)
In fact, authors have earnedover $14 millionwriting, publishing and selling on Leanpub.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
