In modern X11 (using the XInput2 extension), scrolling is not just a series of fake mouse button clicks anymore – it's actually a series of motion events. The mouse wheel is considered to be a relative motion axis; each wheel click usually corresponds to a 15° motion event.
(Touchpads are always handled as motion devices whether you're using them to scroll or not – this distinction is no longer done in hardware, the two-finger scroll gesture is instead recognized by software, most commonly libinput running within Xorg.)
Xorg still emulates a fake button press for every such scroll event, but it's generally up to each toolkit to react to one or the other. Increasingly more programs are using motion events (which allows them to get pixel-precise scrolling from touchpads, instead of jumpy step-scrolling) and will be completely unaffected by changes to the "fake buttons".
So generally, I wouldn't recommend using xmodmap for this at all (as it's entirely based around the legacy X11 "core" input mechanisms). Instead, because your Xorg is most likely using xf86-input-libinput for accessing input devices, just enable the "Natural scrolling" option that it already has.
In Xorg this could be done through XInput2 device properties:
$ xinput
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ AlpsPS/2 ALPS GlidePoint id=11 [slave pointer (2)]
⎜ ↳ HP HP Link-5 micro dongle Mouse id=17 [slave pointer (2)]
⎜ ↳ ...
⎣ Virtual core keyboard id=3 [master keyboard (2)]
↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
↳ Power Button id=6 [slave keyboard (3)]
↳ ...
$ xinput list-props "HP HP Link-5 micro dongle Mouse"
Device 'HP HP Link-5 micro dongle Mouse':
Device Enabled (133): 1
Coordinate Transformation Matrix (135): 1.000000, 0.000000, ...
libinput Natural Scrolling Enabled (276): 0
libinput Natural Scrolling Enabled Default (277): 0
libinput Accel Speed (285): 0.000000
libinput ...
$ xinput set-prop "AlpsPS/2 ALPS GlidePoint" "libinput Natural Scrolling Enabled" 1
(There is also a corresponding Xorg.conf option for this property, see man 4 libinput if you want to enable it permanently.)
Meanwhile, each Wayland compositor has its own configuration – but, most of them in fact use the same libinput library to process the raw input events, and most of them already expose libinput's "natural scrolling" option for touchpad users (it should work just as well for mice, too).
In the worst case it should be a matter of adding a 1-line patch to the compositor's source:
libinput_device_config_scroll_set_natural_scroll_enabled(...);