I'm building a RESTful API that exposes my application's users as users.
My application also features 'documents' and each user has access to specific documents. I'm thinking the natural way to represent that is by exposing the accessible documents through users/{user-id}/documents. 
However, from a usability perspective, it's important for my clients to be able to fetch (and modify) the users that have access to a specific document. Because of that I'm considering 'reversing' this representation to documents/{document-id}/users.
Do these (and especially the latter) seem like proper ways to model this relationship? If I do go with such a solution, how do I model 'granting access to a document'?
I'm leaning towards PUTing a pre-existing user (presumably acquired by GETing users) into documents/{document-id}/users/{user-id}. That seems unsatisfactory however, as I'll be doing an 'update' operation not to actually update the resource but to insert it into a collection. It is especially problematic in terms of semantics as I expect my server-side to ultimately not take into account the complete, sent user representation but rather only cross-reference the id with ids of pre-existing users in order to create an association.
On the other hand, I can't POST into documents/{document-id}/users as I'm not aiming at the creation of a new resource - I specifically don't want one to be created.
Am I doing it wrong?