In the code sample I did:
- reset all margins/paddings to 0
 - set body height to 100%
 - set flex container height to 96%
 - set flex container margin-top to 2%
 
Now this gives me a scroll on the body even if the flex containers height + margin-top only sums up to 98%, so my question is, can't I use margin-top is this way and where does the extra space come from forcing the body to scroll?
Setting the body to overflow:hidden removes the scroll, but that feels more like a band-aid and not considered as a solution (unless this is a "behavior-by design" which needs that in this case).
Edit
Ways like remove the margin-top on the flex container and then set a padding-top: 2%; on the body or use position: relative; top: 2%; on the container or with absolute: position; I can make it work as expected though, but the case here is why margin-top: 2% doesn't do it.
* {
  box-sizing: border-box;
  padding: 0;
  margin: 0
}
html, body {
  background-color: gray;
  height: 100%;
}
.outer {
  display: flex;
  flex-flow: column;
  margin: 0 auto;
  height: 96%;
  width: 50%;
  margin-top: 2%;
}
.top {
  background-color: lightgray;
}
.middle {
  background-color: darkgray;
}
.the-rest {
  flex: 1 0 auto;
  background-color: lightgray;
}
<div class="outer">
  <div class="top">
    top
  </div>
  <div class="middle">
    middle
  </div>
  <div class="the-rest">
    content
  </div>
</div>