There's some problems here. First, you reset the value of factorialNumber2 to something undefined. 
Next problem is that you have a semicolon after the while loop and that the condition for the loop was wrong. Here is working code:
int main()
{
    int number;
    int factorialNumber;
    int counter = 1;
    cout << "Enter a number and I shall find its factorial value.";
    cin >> number;
    factorialNumber=1;
    while(counter <= number)
    {  
        factorialNumber=counter * factorialNumber;
        counter++;
    }
    cout << factorialNumber;
}
I renamed the variables, cause factorialNumber and factorialNumber2 didn't really make sense. Another thing you should think of is to not use using namespace. It will hardly affect you at your level, but it's generally a bad idea. Read why here
Also, a pretty good way to start debugging this would have been just some printouts. Here is an example:
int main()
{
    int number;
    int factorialNumber;
    int counter = 1;
    cout << "Enter a number and I shall find its factorial value.";
    cin >> number;
    factorialNumber=1;
    cout << "Before loop" << endl
         << "number: " << number 
         << " factorialNumber: " << factorialNumber
         << " counter: " << counter << endl;
    while(counter <= number)
    {  
    cout << "Begin loop" << endl
         << "number: " << number 
         << " factorialNumber: " << factorialNumber
         << " counter: " << counter << endl;
        factorialNumber=counter * factorialNumber;
        counter++;
    cout << "End loop" << endl
         << "number: " << number 
         << " factorialNumber: " << factorialNumber
         << " counter: " << counter << endl;
    }
    cout << "After loop" << endl
         << "number: " << number 
         << " factorialNumber: " << factorialNumber
         << " counter: " << counter << endl;
    cout << factorialNumber;
}
It looks a bit crappy, but those printouts are intended to be temporary anyway. Also, of course this is a bit overkill, but it shows the general idea.  
Here is what it would have looked like with your original code:
$ ./a.out 
Enter a number and I shall find its factorial value.5
Before loop
number: -155928352 factorialNumber: -155928352 counter: 1
Begin loop
number: -155928352 factorialNumber: -155928352 counter: 1
End loop
number: -155928352 factorialNumber: -155928352 counter: 2
After loop
number: -155928352 factorialNumber: -155928352 counter: 2
-155928352
With a little experience, you can clearly see what's wrong here. Lastly, use compiler flags to get warnings. Here is an example:
$ g++ yourcode.cpp -Wall -Wextra
yourcode.cpp: In function ‘int main()’:
yourcode.cpp:18:3: warning: this ‘while’ clause does not guard... [-Wmisleading-indentation]
   while(counter == factorialNumber2);
   ^~~~~
yourcode.cpp:19:3: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘while’
   {
   ^