There are two situation to work with libraries for different projects.
Assume the libraries in repoA, and repoB is one of the project repositories.
Situation 1: The libraries only work for you own (only manage libraries in repoA)
You can add the libraries from local repoA to local repoB directly:
In local repoB -> right click the solution for projectB -> Add -> An existing project -> Add the libraries from local repoA to local repoB -> commit -> push.
Now no matter where (in local repoA or local repoB) you changed the libraries, it's actually change the libraries in local repoA, So the libraries in local repoA and local repoB always sync.
Situation 2: other developers also need the libraries in projectB (manage libraries both in repoA and repoB)
If other developers also need to use the libraries, you should also manage the libraries in repoB. The ways usually used are git submodule and git subtree.
Since you have already tries submodules, I will only show git subtree as below:
Add the libraries (assume in master branch of repoA) to repoB:
git subtree add --prefix=repoB <URL for repoB> master
If there has some changes in repoA, get the changes from repoA to repoB:
git subtree pull --prefix=repoB <URL for repoB> master
If make changes for libraries in repoB, to push the changes to repoA:
git subtree push --prefix=repoB <URL for repoB> master