Multi-WAN routers have these two settings:
- Fail-over mode
- Load-balancing mode
The first setting is the simpler of the two: the second WAN link is used only when the first is completely dead.
Load-balancing mode should theoretically include the same fail-over mechanisms, but adds more advanced functionality to use both links simultaneously, spreading traffic evenly across both. This is more complex however, as sometimes when a TCP connection is created across one WAN link, future packets for that connection may need to also travel via that same link, as such the router will store tables in its memory to track this kind of information.
It would make sense to use both connections if you're already paying for them. Just be aware that if one is a lot slower than the other, that some types of traffic that begin on the slower link might remain on the slower link until a new connection is made.
The TP-Link TL-R470T+ Router seems to have both of these features, and with good reviews too. I'd be keen to try one out.
(Thanks also to deveneyi who mentioned a similar thing in their comment.)