Add Total rows with different values of columns

Question

The dataframe looks something like this:

EXT   MARKET       DATE          COL2      COL3
All    100     31/07/2019         10        1
NO     101     31/07/2019         11        0
YES    102     31/07/2019         9         1
NO     100     31/07/2019         8         1
YES    101     31/07/2019         7         0
NO     102     31/07/2019         4         1
All    103     31/07/2019         2         5
.
.
.

I AM WORKING IN DATABRICKS WITH PANDAS BUT I CAN NOT GET THE EXPECTED RESULT

And the expected values should be:

EXT          MARKET      DATE          COL2     COL3
All          100        31/07/2019      10       1
All          101        31/07/2019      10       0
All          103        31/07/2019       2       5
.
.
.
**TOTAL_ALL  (empty)      (empty)        22      6**
YES           102       31/07/2019        9      1
YES           101       31/07/2019        7      0
.
.
.
**TOTAL_YES   (empty)      (empty)       16      1** 
NO            102        31/07/2019      12      60
NO            103        31/07/2019      88      15
.
.
.
**TOTAL_NO     (empty)     (empty)        100     75**

Answer 1

pd.concat

pd.concat([
    d.append({'EXT': f"TOTAL_{k}", **d[['COL2', 'COL3']].sum()}, ignore_index=True)
    for k, d in df.groupby('EXT')
], ignore_index=True)

         EXT  MARKET        DATE  COL2  COL3
0        All   100.0  31/07/2019    10     1
1        All   103.0  31/07/2019     2     5
2  TOTAL_All     NaN         NaN    12     6
3         NO   101.0  31/07/2019    11     0
4         NO   100.0  31/07/2019     8     1
5         NO   102.0  31/07/2019     4     1
6   TOTAL_NO     NaN         NaN    23     2
7        YES   102.0  31/07/2019     9     1
8        YES   101.0  31/07/2019     7     0
9  TOTAL_YES     NaN         NaN    16     1
​