I'm no git expert and it might not be correct but what I'd do is initiate the git repo for the libraries in an external_library, third_party or some folder from which you import to your project. Then when it comes time to update, cd into the folder>project folder and do a git pull origin master or whatever you named it. 
I think it's typical to have git within git i.e.:
myproject
    .git <- root repo for whole project
    application
    system
    third_pary
        cool_library
            .git <- repo for cool_library
            cool_folder
                some_lib.php
        other_cool_library
            .git <- repo for other_cool_library
            neat_libs
                neato.php
From within the root of myproject, git commands will only affect myproject's repo. So
$cd myproject
myproject$ git commit -a -m"neat"
will commit the changes to your myproject repo; all changes in all folders committed (but, if you made a change to /myproject/third_party/cool_library/cool_folder/some_lib.php, you should cd into /myproject/third_party/cool_library/ and do a git commit before committing in /myproject/).
As for pulls:
$cd myproject
myproject$ git pull origin master
will pull into myproject; the other .gits in the third_party folder will be unaffected (unless of course someone else pushed changes to myproject that had changes to the third_party repos. 
So, tl;dr, pulling into your project root will not cause the subfolder repos to pull from their origins. When you need to update, change into the libraries' directory and do a pull there.
In other words, given the above file structure, if there was an update to cool_library that just came out, this is what I'd do:
$cd myproject
myproject$ git commit -a -m"committing before updating cool_library"
myproject$ cd third_party/cool_library
cool_library$ git pull origin master
cool_library$ cd ../../
myproject$ git commit -a -m"commiting after updating cool_library"