If you insist on not using the imresize builtin function, you can use interp2 to rescale the image. First define a scaling factor f. Then you can use interp2 to do it as follows.
% Toy Data
I = im2double( imread( 'bag.png' ) );
% Set Scaling Factor
f = 1/5;
% Resize Image
D = interp2( I, linspace( 1, size(I,2), size(I,2) * f ), linspace( 1, size(I,1), size(I,1) * f )' );
% Plot Image
figure; imshow( I );
figure; imshow( D );
To understand what this code does, first understand that this line finds the number of subdivisions within the linspace.
size(I,2) * f
After you create the linspace, you can use interp2 for the cols for the 2nd argument and rows for the 3rd argument. The image must be a double hence the im2double.
If you don't wish to use interp2 either, as @rayryeng said, you should refer to how to do it without any builtin functions here.