18

We have a couple of ec2 instances on amazon, and we used amazon AIM Linux to set up. The instance type is m1.medium.

Everything is going well until this week.

We found from log file that there would be a huge clock drift happened suddenly, and sometimes there has a 5-minutes discrepancy。

Has anyone else encountered this problem? How can it be dealt with?

starball
  • 20,030
  • 7
  • 43
  • 238
user2028644
  • 181
  • 1
  • 1
  • 3

4 Answers4

10

Time drift can occur due to over utilization of your instances resources. I would encourage you to check you cloudwatch CPU graphs to try identify if your CPU was being over utilized.

If you are running amazon linux AMI, NTP is installed by default.

To verify if NTP is running run

sudo service ntpd status

If it is not installed you can run the following to install it:

sudo yum install ntp

If NTPD is running and your clock time is off I would try to restart it and see if it synchs with the NTP servers your have setup. You can also add more NTP servers by editing the /etc/ntp.conf file. Just search for NTP servers to find ones to add.

Ciaran
  • 1,139
  • 2
  • 11
  • 14
  • 2
    Did you mean `sudo service ntp status` ? – The Internet Jun 11 '14 at 23:00
  • 1
    No when you install ntp it runs as a daemon (ntpd). Example: `sudo service ntpd status ntpd (pid 1312) is running...` `sudo service ntp status ntp: unrecognized service` – Ciaran Jun 12 '14 at 09:39
  • 2
    In Ubuntu the service is `ntp` not `ntpd`. So `sudo service ntp status` for Ubuntu is the right command. – warvariuc Dec 29 '15 at 15:39
  • How offical/reliable is the information stating that time drift can occur due to over utilization of instances resources ? Where did you get this information from ? – user1767316 Jun 13 '23 at 22:53
5

In 2017, Amazon launched the Amazon Time Sync Service. From that announcement post:

You can access the service via the link local 169.254.169.123 IP address. This means you don’t need to configure external internet access and the service can be securely accessed from within your private subnets.

They go on to recommend Chrony instead of ntpd.

B. Kendall
  • 71
  • 1
  • 4
4

Yes, best not to trust the system clock. Just install and configure NTP.

chrskly
  • 959
  • 7
  • 5
  • How to stop the system? Can you give some advice? Thanks! – user2028644 Feb 04 '13 at 02:01
  • I've not used Amazon's own Linux AMIs, but, as far as I'm aware, they're basically RHEL. Here's a guide to setting up NTP on RHEL6: https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sect-Date_and_Time_Configuration-Command_Line_Configuration-Network_Time_Protocol.html – chrskly Feb 04 '13 at 11:07
2

Yes even I faced this issue. Worst part is that I use SES to send emails; and emails sending started failing with the error "Time mismatch".. So, because of AWS one issue; another issue happened.

Restarting the server fixed the problem with immediate effect. Also, create a customer support ticket and let them know. Its a bug on their end. I think this happen with some of their instances. Because of this issue; my business was impacted very adversely hence I even got some dollars credit.

Deepak Singhal
  • 10,568
  • 11
  • 59
  • 98