3

Just posting this here and might help the others:

I got this error when we had an outage a while ago, and tried to restart the slon replication service as the replication is NOT really replicating the data from MASTER to SLAVE:

FATAL  localListenThread: "select "_MY_DATABASE_copy".cleanupNodelock(); insert into "_MY_DATABASE_copy".sl_nodelock values (    1, 0, "pg_catalog".pg_backend_pid()); " - ERROR:  duplicate key value violates unique constraint "sl_nodelock-pkey"
DEBUG2 slon_abort() from pid=xxxx
DEBUG1 slon: shutdown requested
Experiment.Fail.Learn.Repeat.
  • 81
  • 1
  • 8

3 Answers3

2

This, in our environment, is caused by outages (planned or unplanned) in which Postgres goes down before the slon daemons. Oftentimes, restarting the slons doesn't resolve the error. What does resolve it is:

  1. stop all slons which reference the failed/restarted node (if you can, just stop all slons via slon_kill; much simpler).
  2. go into the sl_nodelock table on the master database server, and remove all rows with an nl_nodeid corresponding to the failed/restarted node.
  3. start any stopped slons.
Zac B
  • 3,796
  • 3
  • 35
  • 52
1

I also got this error, and in my case, a third fix applied. We accidentally had a 'slon' process running on the slave-to-be. Our solution was to shut down that process, and then restart the 'slon' processes on the master.

Mark Stosberg
  • 12,961
  • 6
  • 44
  • 49
  • This is similar to the issue I had. We cloned a production node (VM) to snap off a staging database, and whoever did it didn't kill the slon processes when they were done. This, combined with Zac B's solution (although I only deleted the one that was causing an issue), fixed it for me. – Kristopher Jun 10 '13 at 12:47
1

Ok so this is how I fix it, but this doesn't guarantee it will work for everyone:

  • Check the slon process. ~# ps aux | grep slon

  • Then kill all process running the slon replicatin service that will look something like this.

    ~# ps aux | grep slon
root      8321     0.0  0.0   6528  1568 pts/2    S+   10:35   0:00 slon MY_SCHEMA dbname=MY_DATABASE user=postgres host=169.1.1.1 password=password

Normally it has two process running in the background, and this is normal, so we have to kill those process.

~# kill -9 8321

Then run your slon replicatin service script again.

MattH
  • 37,273
  • 11
  • 82
  • 84
Experiment.Fail.Learn.Repeat.
  • 81
  • 1
  • 8