RDMA connection manager isn't mandatory for InfiniBand. It is possible to exchange parameters for an InfiniBand connection over a side-channel (e.g. over a TCP connection on top of IPoIB). For an example you can look at the ibv_rc_pingpong program that is part of the libibverbs library.
However, RDMA connection manager hides some of the complexity of setting up a connection, such as requesting the path information from the subnet manager.
The calls with the rdma_ prefix are part of librdmacm. Most of them are for using the connection manager to set up connections, but some are just wrappers or helper functions for the communication itself.