Linux | Stanimir Genov

The following article is taken from Bryan Kennedy’s blog. Original link: https://plusbryan.com/mysql-replication-without-downtime
I found the information there so valuable that I decided to save it here, as I find myself looking for it frequently.

Replication is something that provides both speed and durability. Usually it is used in environments where the loads on the database are heavy or we want to avoid downtime. However, most methods for replication online include stopping instances, which is unacceptable in a busy production environment.

In order to do it with minimal downtime (just a few restarts):

First, you’ll need to configure the master’s /etc/mysql/my.cnf (or for newer versions of MySQL /etc/mysql/*.conf.d/mysqld.conf) by adding these lines in the [mysqld] section:

server-id=1
binlog-format = mixed
log-bin=mysql-bin
datadir=/var/lib/mysql
innodb_flush_log_at_trx_commit=1
sync_binlog=1

Restart the master mysql server and create a replication user that your slave server will use to connect to the master.

CREATE USER replicant@;
GRANT REPLICATION SLAVE ON . TO replicant@ IDENTIFIED BY '';

Note: Mysql allows for passwords up to 32 characters for replication users.

Next, create the backup file with the binlog position. It will affect the performance of your database server, but won’t lock your tables:

mysqldump --skip-lock-tables --single-transaction --flush-logs --hex-blob --master-data=2 -A > ~/dump.sql

Now, examine the head of the file and jot down the values for MASTER_LOG_FILE and MASTER_LOG_POS. You will need them later:

head dump.sql -n80 | grep "MASTER_LOG_POS"

This file might be huge, so you can gzip it before moving it to the slave (optional):

gzip ~/dump.sql

Now we need to transfer the dump file to our slave server (if you didn’t gzip first, remove the .gz bit):

scp ~/dump.sql.gz mysql-user@:~/

While that’s running, you should log into your slave server, and edit your /etc/mysql/my.cnf (or another position, according to your MySQL version) file to add the following lines:

server-id = 101
binlog-format = mixed
log_bin = mysql-bin
relay-log = mysql-relay-bin
log-slave-updates = 1
read-only = 1

Restart the mysql slave, and then import your dump file (gunzip if you compressed it earlier):

gunzip ~/dump.sql.gz
mysql -u root -p < ~/dump.sql

CHANGE MASTER TO MASTER_HOST='',MASTER_USER='replicant',MASTER_PASSWORD='', MASTER_LOG_FILE='', MASTER_LOG_POS=;
START SLAVE;

To check the progress of your slave:

SHOW SLAVE STATUS \G

If all is well, Last_Error will be blank, and Slave_IO_State will report “Waiting for master to send event”. Look for Seconds_Behind_Master which indicates how far behind it is. It took me a few hours to accomplish all of the above, but the slave caught up in a matter of minutes. YMMV.

And now you have a newly minted MySQL slave server without experiencing any downtime!

A parting tip: Sometimes errors occur in replication. For example, if you accidentally change a row of data on your slave. If this happens, fix the data, then run:

STOP SLAVE;
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
START SLAVE;

I’ve had to give access to multiple accounts to a single folder as a development environment in which it can be tracked who did what. I am writing this down for future reference.

Here was my goal:

Two different accounts, belonging to the same group need to be able to edit each others’ files.
We need to make all files/folders in a location belong to a certain group and then all new files and folders created in that location should also have the same ownership. They should also be group writable. Here is the desired result for newly created files:

-rw-rw-r-- 1 student1 school 0 2007-12-06 22:46 newfile

For this guide our users will be “student1” and “student2” which are both added to the group “school”
Our folder will be at /home/user1/project, so let’s assume user1 is the current owner of the folder and all files and folders in it have the following ownership: user1:user1

The first thing we need to do is change the group ownership of all of the files and folders:

chown -R :school /home/user1/project

After that we need to enable read/write for the user and group for all files and folders. Since we need 775 for folders and 664 for files, we do the following:

find /home/user/project -type d -exec chmod 775 {} \;
find /home/user/project -type f -exec chmod 664 {} \;

Now we need to use GID to set the ownership for all files and folders that will be created in the future (for any subfolder – that’s why it is recursive):

find /home/user/project -type d -exec chmod g+s {} \;

In order to set the access rights for new files and folders we will use ACL. If you don’t have it installed, you can do so by executing (for debian):

apt-get install acl

After installing it, we can set read/write permissions recursively, by defining the modes for owner and group:

setfacl -R -d -m g::rwx -m o::rx /home/user/project

It is important to note that the user’s umask plays a role when creating new files and folders. I.E. the root user in your system will most likely create files with the ownership of root:root and they will not give in to the desired permissions.
For regular users, however this should not be a problem if you have not tinkered with their umask.

That’s it. Result:
Newly created files/folders by user2 will have this ownership: user2:school
Files and folders will be writable by all users in the group, but will keep their original owner.

Category Archives: Linux

MySQL Replication with minimal downtime

Applying default permissions for newly created files within a specific folder