1

I used countvector the obtain the vector of every word in a comment, and used it as the input data of a neural network. However, there is always something wrong with it. The code and the error are as the following:

train_X = vectorizer.transform(train_dataframe['comment'])
valid_X = vectorizer.transform(valid_dataframe['comment'])
test_X = vectorizer.transform(test_dataframe['comment'])
print (train_X.shape)
print (valid_X.shape)
print (test_X.shape)

train_Y = train_dataframe['label'].to_numpy()
valid_Y = valid_dataframe['label'].to_numpy()

train_inputs=train_X
train_targets=train_Y
validation_inputs=valid_X
validation_targets=valid_Y
# Set the input and output sizes
input_size = 31124
output_size = 1
# Use same hidden layer size for both hidden layers. Not a necessity.
hidden_layer_size = 50

# define how the model will look like
model = tf.keras.Sequential([
    # tf.keras.layers.Dense is basically implementing: output = activation(dot(input, weight) + bias)
    # it takes several arguments, but the most important ones for us are the hidden_layer_size and the activation function
    tf.keras.layers.Dense(hidden_layer_size, activation='relu'), # 1st hidden layer
    tf.keras.layers.Dense(hidden_layer_size, activation='relu'), # 2nd hidden layer
    # the final layer is no different, we just make sure to activate it with softmax
    tf.keras.layers.Dense(output_size, activation='sigmoid') # output layer
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

### Training
# That's where we train the model we have built.

# set the batch size
batch_size = 100

# set a maximum number of training epochs
max_epochs = 100


# fit the model
# note that this time the train, validation and test data are not iterable
model.fit(train_inputs, # train inputs
          train_targets, # train targets
          batch_size=batch_size, # batch size
          epochs=max_epochs, # epochs that we will train for (assuming early stopping doesn't kick in)
          validation_data=(validation_inputs, validation_targets), # validation data
          verbose = 2 # making sure we get enough information about the training process
          )  

test_loss, test_accuracy = model.evaluate(test_inputs, test_targets)
print('\nTest loss: {0:.2f}. Test accuracy: {1:.2f}%'.format(test_loss, test_accuracy*100.))

The error is :

 Please provide as model inputs either a single array or a list of arrays. You passed: x=  (0, 1404)    1
  (0, 4453) 2
  (0, 6653) 1
  (0, 8151) 1
  (0, 11070)    1
  (0, 14557)    1
  (1, 817)  1
  (1, 1134) 1
  (1, 1813) 1
  (1, 1827) 1
  (1, 2151) 1
  (1, 4505) 1
  (1, 4647) 1
  (1, 8244) 2
  (1, 8296) 1
  (1, 8332) 1
  (1, 9109) 1
  (1, 9611) 1
  (1, 10080)    1
  (1, 10791)    1
  (1, 11821)    1
  (1, 12714)    1
  (1, 12760)    1
  (1, 13665)    1
  (1, 14349)    1
  : :
  (42423, 16238)    1
  (42423, 17253)    1
  (42423, 18627)    1
  (42423, 19322)    1
  (42423, 19811)    1
  (42423, 21232)    1
  (42423, 23128)    1
  (42423, 25572)    1
  (42423, 25681)    1
  (42423, 27132)    1
  (42423, 27568)    2
  (42423, 27580)    1
  (42423, 27933)    1
  (42423, 30921)    2
  (42424, 932)  1
  (42424, 4078) 1
  (42424, 10791)    1
  (42424, 10835)    1
  (42424, 27628)    1
  (42424, 27933)    1
  (42424, 30220)    1
  (42425, 1813) 1
  (42425, 13868)    1
  (42425, 27580)    1
  (42425, 28749)    1
khelwood
  • 55,782
  • 14
  • 81
  • 108
rene smith
  • 83
  • 1
  • 9

1 Answers1

2

train_inputs is a sparse matrix of type scipy.sparse.csr.csr_matrix as a result of a call to sklearn.feature_extraction.text.CountVectorizer.transform as documented here:
https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html#sklearn.feature_extraction.text.CountVectorizer.transform

You could try to convert the sparse matrix to a dense matrix and to use that as the input for training:

model.fit(train_inputs.toarray().astype(float), ...)

This approach might lead to memory problems on large datasets, though. If you need a more sophisticated approach, you can find more information on how to properly treat sparse matrices with Keras here: Using sparse matrices with Keras and Tensorflow

Andre Wisplinghoff
  • 155
  • 1
  • 5