@chappjc's answer would work; I would like to offer a small change that is a bit more "readable":
First - call ginput without any arguments. It will keep accumulating points clicked until you hit "enter". A little more user friendly.
Second - there's a time and a place for vectorization. When you have just a handful of points (namely, one point per click) it is unlikely that the speedup of vectorized code is worth the pain of sub2ind, repmat, kron...). That leaves us with the following:
imshow(A);
disp( 'Click points in the image; press return when finished' );
[xf, yf] = ginput;
xi = round(xf);
yi = round(yf);
N = numel(xi);
rgbValues = zeros(N, 3);
for ii = 1:numel(xi)
rgbValues(ii,:) = reshape(A(yi(ii), xi(ii), :), 1, 3);
end
This will put the values you want into rgbValues.
Do check that the values of xi and yi are returned in the order shown; I think this is right, but if I'm wrong you would have to use the order A(xi(ii), yi(ii), :) when you read the image).