In this case & takes an object and if the object is not already a Proc, as it is the case with the Symbol :hex it will try to invoke the method to_proc on it. In the Symbol documentation you will find the implementation detail for the to_proc method:
to_proc
Returns a Proc object which respond to the given method by sym.
(1..3).collect(&:to_s) #=> ["1", "2", "3"]
In your case through &:hex the symbol :hex will be converted into the Proc object which is equivalent to { |item| item.hex() }
What is a Proc exactly? Basically the Proc class is a basic anonymous function. In Ruby the notion of a callable object is embodied in Ruby through objects to which you can send the message call. The main representatives of these kind are Proc and Lambda.
Proc objects are self-contained code sequences, that can be created, stored, passed around as method arguments and executed at some point through call. A method like map can also take a block as an argument which is the case if you pass &:hex.
In the method definition of map a kind of implicit call to Proc.new is made using the very same block. Then the Proc is executed through its call method executing the code embodied by the Proc object.