Seems a duplicate of Angular 2: how to apply limit to *ngFor?
You can use Array.slice(...) do something like that if you don't want to use <template>: 
<div class="chat-window-container">
    <chat-window
            *ngFor="let box of (windows ? windows.slice(0,2): []); let i = index"
            [box]="box">
    </chat-window>
</div>
EDIT : show the last two boxes (eg.: if 3 or more boxes):
You simply have to enhance a bit the Array.slice(...) parameters to make them more dynamic. This way, you'll always show the two latest boxes.
<div class="chat-window-container">
    <chat-window
            *ngFor="let box of (windows ? windows.slice(windows.length-2,2): []); let i = index"
            [box]="box">
    </chat-window>
</div>
For example, if you have 4 boxes in windows, it will be the same that:
<div class="chat-window-container">
    <chat-window
            *ngFor="let box of (windows ? windows.slice(2,2): []); let i = index"
            [box]="box">
    </chat-window>
</div>