As I was writing this question I found an answer. Unfortunately I wasn't able to find a way to solve this problem in the GUI, but at the command line I was able to run:
gsettings set org.mate.Marco.general action-middle-click-titlebar none
to fix the problem.
It works as follow:
gsettings set = command to change a "g" ie. GNOME (ie. MATE) setting
org.mate.Marco.general = "schema" (ie. settings category) for general MATE UI settings
action-middle-click-titlebar = setting that controls what happens when a window's title bar gets middle-clicked
You then have have several options you can set it to:
toggle_shade
toggle_maximize
toggle_maximize_horizontally
toggle_maximize_vertically
minimize
none = what I set it to to fix things (by making middle clicks do nothing)
lower = what I had it set to: makes the window "lower" itself behind the window behind it (similar to ALT + TAB)
menu
last