You need to iterate over your input dictionary; you don't have input rows at the ready here.
For each key-value pair in your input dictionary, you have multiple rows. The first row consists of the key, and the first value, and then you have more rows for the remaining values with the first column empty.
You need to write those rows explicitly:
csv_writer = csv.writer(csv_file, dialect='excel')
for key, values in d.items():
csv_writer.writerow((key, values[0]))
csv_writer.writerows([(None, value) for value in values[1:]])
I used None for the empty column value, that results in an empty column in the output:
>>> import csv
>>> from io import StringIO
>>> output = StringIO()
>>> d = {'a': [1, 3, 4], 'b': [2, 3, 4], 'c': [1]}
>>> csv_writer = csv.writer(output, dialect='excel')
>>> for key, values in d.items():
... csv_writer.writerow((key, values[0]))
... csv_writer.writerows([(None, value) for value in values[1:]])
...
5
5
5
>>> print(output.getvalue())
a,1
,3
,4
b,2
,3
,4
c,1
You probably want to add a sorted() call to the d.items() loop; otherwise your output will be in dictionary order, which is arbitrary based on insertion and deletion history, at least before Python 3.6.