When you want to calculate the distance between points with latitude/longitude coordinates, the distm function from the geosphere package gives you several methods: distCosine, distHaversine, distVincentySphere & distVincentyEllipsoid. Of these, the distVincentyEllipsoid is considered the most accurate one. In these answers I showed how to calculate the distance between two different lists of points:
However, your case is a bit different as you want to compare within a list of points with coordinates. By slightly changing the method I showed in these answers, you can achieve the same. An illustration on how to do that with the data you provided:
The data:
points <- structure(list(p = structure(1:5, .Label = c("A", "B", "C", "D", "E"), class = "factor"),
                         lon = c(100.113, 99.8961, 99.8829, 101.2457, 102.1314), 
                         lat = c(17.5406, 20.0577, 20.0466, 16.8041, 19.8881)), 
                    .Names = c("p", "lon", "lat"), class = "data.frame", row.names = c(NA, -5L))
Note that I added a variable p with names for the points.
Original method:
First you create a distance matrix with:
distmat <- distm(points[,c('lon','lat')], points[,c('lon','lat')], fun=distVincentyEllipsoid)
which gives:
> distmat
         [,1]       [,2]       [,3]     [,4]     [,5]
[1,]      0.0 279553.803 278446.927 145482.3 335897.8
[2,] 279553.8      0.000   1848.474 387314.3 234708.0
[3,] 278446.9   1848.474      0.000 386690.7 235998.8
[4,] 145482.3 387314.334 386690.666      0.0 353951.5
[5,] 335897.8 234707.958 235998.784 353951.5      0.0
When you now assign the nearest point to each point with:
points$nearest <- points$p[apply(distmat, 1, which.min)]
each point will be assigned to itself as nearest point:
> points
  p      lon     lat nearest
1 A 100.1130 17.5406       A
2 B  99.8961 20.0577       B
3 C  99.8829 20.0466       C
4 D 101.2457 16.8041       D
5 E 102.1314 19.8881       E
Adaptation:
You can prevent that behavior by replacing the the 0 values in the distance matrix distmat with:
distmat[distmat==0] <- NA
When you now assign the nearest point to each point with:
points$nearest <- points$p[apply(distmat, 1, which.min)]
you get the correct values:
> points
  p      lon     lat nearest
1 A 100.1130 17.5406       D
2 B  99.8961 20.0577       C
3 C  99.8829 20.0466       B
4 D 101.2457 16.8041       A
5 E 102.1314 19.8881       B