The associations could be set up thus:
#app/models/user.rb
Class User < ActiveRecord::Base
    has_many :comments
    has_and_belongs_to_many :mentions, join_table: "comments_users", association_foriegn_key: "comment_id"
end
Class Comment < ActiveRecord::Base
    belongs_to :author, class_name: "User", foreign_key: "author_id"
    has_and_belongs_to_many :mentions, join_table: "comments_users", foreign_key: "comment_id"
end
#comments_users
comment_id | user_id
This will allow you to call:
@user.comments #-> shows comments user has authored
@user.mentions.first.comment #-> shows first comment user was mentioned in
@comment.author #-> shows user who authored comment
@comment.mentions.first.user #-> shows first user who was mentioned in comment
Update
HABTM still needs a table (Rails migration for has_and_belongs_to_many join table), but the difference is that it doesn't need a primary key column (just comment_id | user_id)
We've created a "self-referential" habtm relationship, meaning you don't need to "create" any records -- they should all be created already. The HABTM will just reference them. As such, you'll need to use the << ActiveRecord method to add records into your mentions collection:
#app/controllers/comments_controller.rb
Class CommentsController < ActiveRecord::Base
     def create
         @comment = Comment.new(comments_params)
         @comment.save_with_mentions
     end
end
#app/models/comment.rb
Class Comment < ActiveRecord::Base
    def save_with_mentions
        comment = self.save
        #do something to find mentioned users
        mentioned_users = User.where("user_id = 1") #example
        for user in mentioned_users do
            comment.mentions << user
        end
    end
end