Keras Realtime Augmentation adding Noise and Contrast

Question

Keras provides an ImageDataGenerator class for realtime augmentation, but it does not include contrast adjustment and addition of noise.

How can we apply a random level of noise and a random contrast adjustment during training? Could these functions be added to the 'preprocessing_function' parameter in the datagen?

Thank you.

score 12 · Answer 1 · answered Apr 25 '20 at 12:01

12

You could indeed add noise with preprocessing_function.

Example script:

import random
import numpy as np

def add_noise(img):
    '''Add random noise to an image'''
    VARIABILITY = 50
    deviation = VARIABILITY*random.random()
    noise = np.random.normal(0, deviation, img.shape)
    img += noise
    np.clip(img, 0., 255.)
    return img

# Prepare data-augmenting data generator
from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(
        rescale=1./255,
        rotation_range=40,
        width_shift_range=0.2,
        height_shift_range=0.2,
        zoom_range=0.2,
        preprocessing_function=add_noise,
    )

# Load a single image as our example
from keras.preprocessing import image
img_path = 'cat_by_irene_mei_flickr.png'
img = image.load_img(img_path, target_size=(299,299))

# Generate distorted images
images = [img]
img_arr = image.img_to_array(img)
img_arr = img_arr.reshape((1,) + img_arr.shape)
for batch in datagen.flow(img_arr, batch_size=1):
    images.append( image.array_to_img(batch[0]) )
    if len(images) >= 4:
        break

# Display
import matplotlib.pyplot as plt
f, xyarr = plt.subplots(2,2)
xyarr[0,0].imshow(images[0])
xyarr[0,1].imshow(images[1])
xyarr[1,0].imshow(images[2])
xyarr[1,1].imshow(images[3])
plt.show()

Example images generated by the script:

answered Apr 25 '20 at 12:01

Andriy Makukha

7,580
1
38
49

2

Nice answer. Note this noise as implemented will only brighten parts of the image, never darken. The other note is that your ImageDataGenerator will be producing images with pixel values on [0 1]. I think a lot of CNN-based techniques expect zero-mean data - so you'd need to shift this by -.5 if your data was mean .5 (as many large image collections would be, under the transformation you list). I do seem to remember that if you use a transfer learning technique (like starting with a pre-trained inception network, etc) some expect zero-mean data. – michar Jun 21 '20 at 22:34
2

One more note: if you use ImageDataGenerator to do a train/validation split (validation_split=%), I think it *may* apply that preprocessing_function to the validation set as well. Which is maybe something you do not want to happen. – michar Jun 21 '20 at 22:48
@michar, good points. Though, because the mean is zero, on average, it will lighten parts of the image which are close to black and darken parts of the image which are close to white. – Andriy Makukha Jun 22 '20 at 00:46
Yes - but I think since you are working with image pixels on [0 255], there will be an issue, as the mean there is 127.5. Consider using "noise = np.random.normal(127.5, deviation, img.shape)' and a final line of 'return img - 127.5'. Them after the ImageDataGenerator scales by 1/255, the result will be on [-.5, .5] with zero mean. Does that make sense? – michar Jun 22 '20 at 03:31
@michar, yes, I agree that it should probably be centered at zero. I just clarified about "never darken", because the highlights are clearly darkened by the noise. Thanks for the comments anyway. – Andriy Makukha Jun 22 '20 at 04:48

score 2 · Accepted Answer · answered Apr 13 '17 at 01:37

2

From the Keras docs:

preprocessing_function: function that will be implied on each input. The function will run before any other modification on it. The function should take one argument: one image (Numpy tensor with rank 3), and should output a Numpy tensor with the same shape.

So, I created a simple function and then used the image augmentation functions from the imgaug module. Note that imgaug requires images to be rank 4.

answered Apr 13 '17 at 01:37

Chris Parry

2,937
7
30
71

Is it possible to make the `preprocessing_function` affect the `y_train` data just as much as the `x_train` ? I'm dealing with images in each set, and if I flip X, I would like Y to be flipped too, for example. – payne Sep 25 '18 at 02:20
Please share the code you created, it would be very helpful for others :) – NeStack Aug 05 '19 at 14:48

NeStack · Answer 3 · 2019-08-05T15:21:51.663

I found in this blog that you can do something as simple as:

from keras.layers import GaussianNoise

model.add(Dense(32))
model.add(GaussianNoise(0.1))
model.add(Activation('relu'))
model.add(Dense(32))
...

Unfortunately, I can't find an analogous way to adjust/augment the contrast. But you can, according to this post, augment the brightness with

from keras.preprocessing.image import ImageDataGenerator
ImageDataGenerator(brightness_range=[range_min,range_max])

Keras Realtime Augmentation adding Noise and Contrast

3 Answers3

Linked