If you don't care whether the headers have borders around them and bold font, and you don't want to restrict the version of openpyxl, the quickest way is to overwrite the header_style dictionary to be None.
If you also have dates or datetimes, you must also explicitly set the workbook's date and datetime formats to None:
from datetime import datetime
import pandas as pd
pd.core.format.header_style = None  # <--- Workaround for header formatting
dt = datetime.now()
d = datetime.date(datetime.now())
df1 = pd.DataFrame([{'c1': 'alpha', 'c2': 1}, {'c1': 'beta', 'c2': 2}])
df2 = pd.DataFrame([{'c1': dt, 'c2': d}, {'c1': dt, 'c2': d}])
with pd.ExcelWriter('output.xlsx') as writer:
    writer.date_format = None # <--- Workaround for date formatting
    writer.datetime_format = None  # <--- this one for datetime
    df1.to_excel(writer,'Sheet1')
    df2.to_excel(writer,'Sheet2')
Not clear why the keyword arguments never make it through the openpyxl deprecation wrapper...but they don't.  In addition if you're formatting any other cells use the new openpyxl api.
All of these problems go away if you have Anaconda, by the way.