How can I generate a ggplot2 scatterplot of two groups with the means indicated together with X and Y error bars, like this?

Here is a reduced example (using dput to recreate the data.frame df) with two groups of cells and three measures, and I'd like to say plot Peak against Rise, or Peak against Decay. That much is straightforward, but I would like to add points indicating the group means with X and Y error bars (+/- sem).
Is there a way to do this within ggplot2, or do I need to generate means and sem values first? This post draw my attention to geom_errorbarh but I'm still uncertain as to the best way to proceed.
library(ggplot2)
df<-structure(list(Group = c("A", "A", "A", "A", "A", "A", "A",
"A", "B", "B", "B", "B", "B", "B", "B", "B"), Peak = c(102.975,
37.805, 64.996, 66.36, 199.354, 7.425, 34.137, 366.59, 10.165,
14.833, 702.525, 39.086, 8.286, 122.783, 105.762, 37.018), Rise = c(0.346855,
0.24165, 0.24028, 0.461548, 0.194016, 0.164047, 0.484375, 0.307861,
0.438538, 0.488083, 0.549423, 0.365448, 0.511551, 0.33596, 0.331467,
0.270096), Decay = c(1.3874, 1.07407, 1.88787, 2.64408, 1.1462,
0.615963, 4.04641, 1.48701, 3.61397, 4.1838, 1.92746, 3.64329,
4.21354, 0.812695, 1.14611, 1.28279)), .Names = c("Group",
"Peak", "Rise", "Decay"), class = "data.frame", row.names = c(NA,
-16L))
ggplot(df, aes(Peak, Rise)) +
geom_point(aes(colour=Group)) +
theme_bw(14)
I have tried something like:
library(doBy)
sem <- function(x) sqrt(var(x)/length(x))
z<-summaryBy(Peak+Rise+Decay~Group, data=df, FUN=c(mean,sem))
z
to get the values, but easily (and flexibly) incorporating them into the ggplot code is defeating me.
