You need to set your root element and body to take up 100% of the height.
Explanation:
Your parent box had a height of 100%, that is the full height of its parent - the body. You hadn't set a height on the body, and its initial height is auto. Therefore your height was auto, only taking up the space needed to fit the content and padding.
* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
}
html,
body {
  height: 100%;
}
.parent-box {
  position: relative;
  background-color: red;
  width: 100%;
  height: 100%;
  padding: 2rem;
  top: 0;
  left: 0;
}
.child-box {
  position: absolute;
  width: 100%;
  height: 100%;
  background-color: green;
  top: 0;
  left: 0;
}
<div class="parent-box">
  <div class="child-box">
    <h4>child box</h4>
  </div>
</div>
 
 
Here is an improved solution with less code:
Width 100% is default for block elements, inset can be used for the child div instead of height and top/left, top and left have no effect on relative positioned elements so remove them from the parent.
* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
}
html,
body {
  height: 100%;
}
.parent-box {
  position: relative;
  background-color: red;
  height: 100%;
  padding: 2rem;
}
.child-box {
  position: absolute;
  background-color: green;
  inset: 0;
}
<div class="parent-box">
  <div class="child-box">
    <h4>child box</h4>
  </div>
</div>