You are trying this:
STRING_AGG('', @delimiter, '', [Column] )   FROM CTE  GROUP BY [Column]
which means you want to concatenate the values in the column, but grouping by the same values. So, you will get only one value.
From here, first remove the GROUP BY and use the correct syntax:
STRING_AGG([Column],@delimiter) FROM CTE
Also, pay attention to this section from the above linked documentation:
Return Types
Return type depends on first argument (expression). If input argument is string type (NVARCHAR, VARCHAR), result type will be same as input type. The following table lists automatic conversions:
| Input expression type | 
Result | 
| NVARCHAR(MAX) | 
NVARCHAR(MAX) | 
| VARCHAR(MAX) | 
VARCHAR(MAX) | 
| NVARCHAR(1...4000) | 
NVARCHAR(4000) | 
| VARCHAR(1...8000) | 
VARCHAR(8000) | 
| int, bigint, smallint, tinyint, numeric, float, real, bit, decimal, smallmoney, money, datetime, datetime2, | 
NVARCHAR(4000) | 
 
If you are expecting the concatenated string to exceed VARCHAR(8000) or NVARCHAR(4000), you need to cast the value like this:
STRING_AGG(CAST([Column] AS VARCHAR(MAX)),@delimiter) FROM CTE