@Frank Nielsen answer is correct, but i wanted to add one more method if someone else wanted different approach.
Part with creating certificate is the same as on the blog that @Franck Nielsen posted link to it. Just in this approach all configuration is done automatically through appsettings.json.
ASP.NET Core 5.0 docs says:
CreateDefaultBuilder calls Configure(context.Configuration.GetSection("Kestrel")) by default to load Kestrel configuration.
So you should add "Kestrel" section to appsetting.json
{
  "Kestrel": {
    "Endpoints": {
      "Http": {
        "Url": "http://localhost:5000"
      },
      "Https": {
        "Url": "https://localhost:5001",
        "Certificate": {
          "Path": "<path to .pfx file>",
          "Password": "<certificate password>"
        }
      }
    }
  }
}
Program.cs could look like this with no additional configuring of Kestrel.
public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        })
        .UseWindowsService();
And viola, rest is done by framework...
I found this method more cleaner and there is no hustle with things like getting the root directory of application .exe in windows service.
Link to ASP.NET Core 5.0 docs: Configure endpoints for the ASP.NET Core Kestrel web server