Categories
How-To Sys Admin

Backup to AWS S3 Bucket

While this is not an uncommon thing to do, I couldn’t find a straight forward example for both databases and file directories. So of course, I had to write my own (albeit based on a database only script from mittsh). For the TLDR; just go to https://github.com/mikemcmurray/backup-to-s3

It’s written in python using the ubiquitous boto3 and just reads the config and source databases and directories from a JSON configuration file. In probably less than 5min you can have completed your first backup and then just schedule it from then on out.

NOTE: The use of S3 incurs a cost. You are solely responsible for managing the use of that system and any costs incurred.

Installation

Copy the files in this repo or just “git clone” them to the machine that needs backing up. The following will clone the current script into a new folder.

git clone https://github.com/mikemcmurray/backup-to-s3.git backup-to-s3

Change into that new folder and install the libraries listed in the requirements.txt file. i.e. “pip install boto3 argparse –user”

Rename and change the config file to suit your own needs. Run the script manually to make sure your config is working as expected.

If all is good then add it to your crontab to run as often as you like. Each backup file is named with the current timestamp to the second so multiple backups each day can be identified.

Run the backup as below. Full paths defined if you’re putting it into crontab and based on a Ubuntu machine layout. User home is ubuntu in this example as that’s the defualt user name on AWS Ubuntu instances.

/usr/bin/python /home/ubuntu/backup-to-s3/backup-to-s3.py /home/ubuntu/backup-to-s3/backup-to-s3.json

You can use the AWS S3 key values in the config to split different backups up into S3 keys (like folders) based on your server names or client accounts, etc.

S3 and Glacier

If you have a heap of data in S3 it will start to cost you more than a coffee a day to keep there. But AWS offer cheaper, longer-term storage in another product called Glacier. The nice thing about these two products is that the bucket properties in S3 can automatically “age out” files from S3 into Glacier. So then you only keep the very new backups in S3 and the rest end up in Glacier where a few hundred GB only costs you a coffee per month.

Categories
How-To Sys Admin

MySQL Fails to Update on Ubuntu

So your Ubuntu server doesn’t want to upgrade MySQL using apt-get and fails with the following error?

mysql_upgrade: Got error: 1045: Access denied for user 'debian-sys-maint'@'localhost' (using password: YES) while connecting to the MySQL server
Upgrade process encountered error and will not continue.

Thankfully the fix should be fairly easy to carry out. For some reason the password for the MySQL user debian-sys-maint has got out of sync in the MySQL database compared to that stored in /etc/mysql/debian.cnf.

Get the password that the update process thinks is in use from that file. You’ll need to do this as the root user.

grep 'password' /etc/mysql/debian.cnf

The same password should be echo’d twice. Copy and paste the password somewhere safe – like a password manager tool.

Log into MySQL as root from the command line using the normal method below. You will need to use the password for the MySQL root user here when prompted.

mysql -u root -p

Reset the password for the debian-sys-maint user, making sure you substitute in your own password from above.

GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY 'password-here';

Now if you run the upgrade process again, it should progress and complete any MySQL server upgrades as needed.

Categories
How-To Sys Admin

MySQL Replication on Ubuntu with DRBD

I’ve been looking around for some easy and open-source ways to handle database replication for a handful of small but important MySQL databases. A few options were viable but usually included too many config changes for things like creating a new database. DRDB on a Linux server seems to be one of the fastest and easiest methods to handle database synchronisation for DR purposes, so this is the subject of this post. The content is a combination of two main sources from Mark Schoonover and the Ubuntu server guide and the gotchas I found along the way.

This post will show you how to create two MySQL servers that automatically replicate all their databases using DRBD. With Heartbeat installed on a third machine you’ll have basic fail over protection as well (we’ll do this in another post). Only one of the database servers will be active at any one time.

Categories
Sys Admin

Installing a Windows Dev Box

In the interest of saving this for future reference, here’s is a general how-to for installing a Windows based Web development server.

The reason for running up a Windows server as opposed to the typical LAMP alternative on Windows was due to frustration. The older Ubuntu server I had just made it very difficult to install the versions of PHP and MySQL that I wanted.

Applications I used were:
Apache 2.0.55
PHP 4.4.2
MySQL 4.1.18
Subverison 1.3