Skip to main content

Database Replication

I spent some time yesterday researching (free) database replication options. Judging from the newsgroup posts I saw, there's a lot of confusion out there. The most common use case appears to be failover, i.e., you want to minimize downtime in the face of software or hardware failure by replicating your data across multiple machines. But, the most commonly-used options are completely inappropriate for this purpose.

As Josh Berkus explained, there are two "dimensions" to replication: synchronous vs async, and master/slave vs multimaster.

For a failover solution, if you want database B to take over from database A in case of failure, with no data loss, only synchronous solutions make sense. By definition, asynchronous replication means that database A can commit a transaction before those changes are also committed to database B. If A happens to fail between commit and replication, you've lost data. If that's not acceptable for you, then neither is async replication.

Be aware that the most popular replication solutions for both PostgreSQL and MySQL are asynchronous.

  • In part because of the contributions by the likes of Fujitsu and Affilas (.org and .info registrar), Slony-I is the most high-profile replication solution for PostgreSQL. Slony-I provides only asynchronous replication.
  • MySQL replication is also asynchronous.

So what are the options for synchronous replication?

  • MySQL "clustering" appears to allow for synchronous replication, but requires use of the separate NDB storage engine, which has a long list of limitations vs MyISAM or InnoDB. (No foreign key support, no triggers, basically none of the features MySQL has been adding for the past few years. Oh, and you need enough RAM to hold your entire database twice over.)
  • PgCluster for postgresql seems fairly mature, but the 1.3 (8.0-based) and 1.5 (8.1-based) versions still aren't out of beta. PgCluster also patches the postgresql source directly, which makes me a little nervous.
  • Another option is something like pgpool, which multiplexes updates across multiple databases. The biggest limitation of this approach is that you're on your own for recovery, i.e., after A goes down and you switch to B alone, how do you get A back in sync? A fairly common approach is to combine pgpool with Slony-I async replication for recovery.

The bottom line is, high availability isn't as simple as adding whatever "replication" solution you first run across. You need to understand what the different kinds of replication are, and which are appropriate to your specific situation.

Comments

Anonymous said…
If you have not already, check out Linux Virtual Server (LVS). I've used LVS with MySQL before for high availability and load balancing. By using keepalived and some scripts any slave could take over as master if the master heartbeat died.
5:15 PM
Jonathan Ellis said…
So, basically the same approach as pgpool, but lower-level? Interesting.
7:49 PM
Anonymous said…
How is pgpool+Slony accomplished, exactly? Are you referring to placing pgpool in master/slave mode as described at the bottom of the pgpool Web site?
7:25 PM
Jonathan Ellis said…
Yes, in which case (unlike what I implied) you're relying entirely on slony for the replication so you're back to async. Sorry for the mistake.
8:21 PM
Anonymous said…
which one do you prefer between asynchronous and synchronous with pgcluster and slony?? and why??
9:37 AM
Jonathan Ellis said…
We use Slony; it has a much bigger user base and is the "safe" choice. And if you're doing replication that probably matters to you. :)
10:06 AM
Anonymous said…
i want to try to use slony to replicate my dbase..could you help me give me simple way thats i can follow?? i've try to use like in documentation but i still can't do it.i am novice user who want to know performance slony.could you give me the site or url which told me the difference between slony and pgcluster.or may be,how can i measure performance slony?? i mean how i can measure time of replication in slony??
Thanks
Regards,
Bayu
by_pacitan@yahoo.com
9:53 AM
Post a Comment

Popular posts from this blog

PyCon Python IDE review

I presented an IDE review at PyCon last Friday. It was basically a re-review of what I thought were the 3 most promising IDEs from the Utah Python User Group IDE review , to which I added SPE, which was by far the most popular of the ones we left out that time. The versions reviewed are: PyDev 1.0.2 SPE 0.8.2.a Komodo 3.5.2 Wing IDE 2.1 beta 1 I'd intended to base my presentation around a comparison of writing a smallish program in each of the IDEs, but the more I tried to make this not suck, the more I realized it was a losing proposition. Instead, I decided to try to focus on the features in each that most set them apart from the others (both positive and negative); this seemed more likely be useful. (I did a new feature matrix for this review, which is included after my comments. The slides I used are also up, at http://utahpython.org/jellis/pycon-ides.pdf , but aren't very useful absent video of the presentation itself. Hence this post.) PyDev PyDev has g...
43 comments
Read more

Troubleshooting the ps3 wireless network connection, including error 80130128

My father got a ps3 for Christmas, but ran into some problems getting it on his wireless network. The first one was "connection error 80130128" after configuring it to use DHCP. I couldn't google anything useful about this; just a few other hapless victims asking if anyone had any ideas. Fortunately Dad had his laptop there too and noticed Windows complaining that two machines on the network were both using the same IP. So, over the phone, I walked him through setting up the ps3 with a static address: on his laptop, run -> cmd ipconfig Read the "gateway" ip. Put that into his browser to go to his router's admin page Find the DHCP settings for his router to see what range of IPs it hands out; pick one outside that range Set up the ps3 with that IP, the router IP as primary dns, and an opendns server as secondary This made the connection test happy. But when he tried to go to the playstation store, it gave a DNS error. If he repeated the connection te...
7 comments
Read more

A review of 6 Python IDEs

(March 2006: you may also be interested the updated review I did for PyCon -- http://spyced.blogspot.com/2006/02/pycon-python-ide-review.html .) For September's meeting, the Utah Python User Group hosted an IDE shootout. 5 presenters reviewed 6 IDEs: PyDev 0.9.8.1 Eric3 3.7.1 Boa Constructor 0.4.4 BlackAdder 1.1 Komodo 3.1 Wing IDE 2.0.3 (The windows version was tested for all but Eric3, which was tested on Linux. Eric3 is based on Qt, which basically means you can't run it on Windows unless you've shelled out $$$ for a commerical Qt license, since there is no GPL version of Qt for Windows. Yes, there's Qt Free , but that's not exactly production-ready software.) Perhaps the most notable IDEs not included are SPE and DrPython. Alas, nobody had time to review these, but if you're looking for a free IDE perhaps you should include these in your search, because PyDev was the only one of the 3 free ones that we'd consider using. And if you aren...
36 comments
Read more