There are multiple scenarios for why this might happen.
First off, make sure your jack is fully inserted into the plug. It could be a loose connection.
Second, you have multiple places where you can configure the output volume, and if one of them is turned down you will experience low volume output even if the other controls are turned to the max.
Start up a media player or radio broadcast/YouTube video to make sure you have sound playing so you can test this along the way. Also connect your headset so you can hear if the volume changes (I'd recommend you leave them out, so you don't get your ear damaged if they suddenly play very loud).
Make sure the volume control in your media player/radio/YouTube is set to maximum volume. These volume controls work independent of the system volume, so you might need to turn them both up.
Using your hardware controls (which will be either a keyboard shortcut or buttons above your keyboard), turn your volume up to somewhere between 50 and 100% volume.
Next go to the windows mixer and check there for the volume settings. Click the speaker icon in the system tray (usually far right side of the task bar) and click "Mixer" make sure all the outputs are turned way up. You should see the output wave moving for the output device which is currently outputting sound.
If you're still not getting enough volume check the playback device settings (right click the speaker icon and click "playback devices", go to the "Communication" tab and select "Do nothing" in the radio buttons) and see if that helps. If it does, you may have some application running, which Windows is registering as communicating, which can be set to turn your playback volume down.