Custom Serilog sink with injection?

Question

I have create a simple Serilog sink project that looks like this :

namespace MyApp.Cloud.Serilog.MQSink
{
    public class MessageQueueSink: ILogEventSink
    {
        private readonly IMQProducer _MQProducerService;
        public MessageQueueSink(IMQProducer mQProducerService)
        {
            _MQProducerService = mQProducerService;
        }
        public void Emit(LogEvent logEvent)
        {
            _MQProducerService.Produce<SendLog>(new SendLog() { LogEventJson = JsonConvert.SerializeObject(logEvent)});
        }
    }
}

The consuming microservice are starting up like this :

        var configurationBuilder = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();
        var appSettings = configurationBuilder.Get<AppSettings>();

        configurationBuilder = new ConfigurationBuilder().AddJsonFile("ExtendedSettings.json").Build();

            Host.CreateDefaultBuilder(args)
                .UseMyAppCloudMQ(context => context.UseSettings(appSettings.MQSettings))
                .UseSerilog((hostingContext, loggerConfiguration) => loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration))
                .ConfigureServices((hostContext, services) =>
                {
                    services
                        .AddHostedService<ExtendedProgService>()
                        .Configure<MQSettings>(configurationBuilder.GetSection("MQSettings"))
                })
                .Build().Run();

The serilog part of appsettings.json looks like this :

  "serilog": {
    "Using": [ "Serilog.Sinks.File", "Serilog.Sinks.Console", "MyApp.Cloud.Serilog.MQSink" ],
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    },
    "Enrich": [ "FromLogContext", "WithMachineName", "WithProcessId" ],
    "WriteTo": [
      {
        "Name": "MessageQueueSink",
        "Args": {}
        }
    ]
  }

The MQSink project is added as reference to the microservice project and I can see that the MQSink dll ends up in the bin folder.

The problem is that when executing a _logger.LogInformation(...) in the microservice the Emit are never triggered, but if I add a console sink it will output data? I also suspect that the injected MQ will not work properly?

How could this be solved?

EDIT :

Turned on the Serilog internal log and could see that the method MessageQueueSink could not be found. I did not find any way to get this working with appsetings.json so I started to look on how to bind this in code.

To get it working a extension hade to be created :

public static class MySinkExtensions
    {
        public static LoggerConfiguration MessageQueueSink(
                  this Serilog.Configuration.LoggerSinkConfiguration loggerConfiguration,
                  MyApp.Cloud.MQ.Interface.IMQProducer mQProducer = null)
        {
            return loggerConfiguration.Sink(new MyApp.Cloud.Serilog.MQSink.MessageQueueSink(mQProducer));
        }
    }

This made it possible to add the custom sink like this :

Host.CreateDefaultBuilder(args)
                    .UseMyAppCloudMQ(context => context.UseSettings(appSettings.MQSettings))
                     .ConfigureServices((hostContext, services) =>
                    {
                        services
                            .Configure<MQSettings>(configurationBuilder.GetSection("MQSettings"))
                    })
                    .UseSerilog((hostingContext, loggerConfiguration) => loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration).WriteTo.MessageQueueSink())
                    .Build().Run();

The custom sink is loaded and the Emit is triggered but I still do not know how to inject the MQ in to the sink? It would also be much better if I could do all the configuration of the Serilog and sink in the appsettings.json file.

Answer 1

In order to let the dependency injection flow handle all this, you can use the ReadFrom.Services extension method.
This extension method is part of the Serilog.Extensions.Hosting NuGet package, which is also bundled with the Serilog.AspNetCore NuGet package.

From the method description

Configure the logger using components from the services.
If present, the logger will receive implementations/instances of Serilog.Core.LoggingLevelSwitch, Serilog.Core.IDestructuringPolicy, Serilog.Core.ILogEventFilter, Serilog.Core.ILogEventEnricher, Serilog.Core.ILogEventSink, and Serilog.Configuration.ILoggerSettings.

So if you register that MessageQueueSink into the DI container, it will get injected into Serilogs Logger and on its turn will also have its own dependencies injected (here: an IMQProducer instance).
This will happen for the static Serilog.Log.Logger as also for any ILogger<T> instances that get injected somewhere.

Note that this setup does not require/use an extension method upon LoggerSinkConfiguration.

Find below the code parts.

---

Setup of the DI container and Serilog

class Program
{
    static void Main(string[] args)
    {
        using var host = CreateHostBuilder(args);
        host.Services.GetService<IMyService>().Run();
        host.Run();
    }

    public static IHost CreateHostBuilder(string[] args)
        => Host.CreateDefaultBuilder(args)
                .ConfigureServices((hostBuilderContext, services) => 
                {
                    services.AddTransient<ILogEventSink, MessageQueueSink>();
                    services.AddTransient<IMQProducer, MQProducer>();
                    // ...
                    services.AddTransient<IMyService, MyService>();
                })
                .UseSerilog((context, services, configuration) 
                    => configuration
                        .ReadFrom.Services(services)
                        .ReadFrom.Configuration(context.Configuration)
                )
                .Build();
}

For completeness the other classes mentioned above:

The sink

public class MessageQueueSink : ILogEventSink
{
    private readonly IMQProducer _mqProducer;

    public MessageQueueSink(IMQProducer mqProducer)
        => _mqProducer = mqProducer;
    
    public void Emit(LogEvent logEvent)
        => _mqProducer.Produce(logEvent);
}

The other services/dependencies mentioned above

// Stub for the real MQ stuff.
public interface IMQProducer
{
    void Produce(LogEvent logEvent);
}

public class MQProducer : IMQProducer
{
    public void Produce(LogEvent logEvent)
        => Console.WriteLine($">>> Producing log event: {logEvent.RenderMessage()}");
}   

public interface IMyService
{
    void Run();
}

public class MyService : IMyService
{
    private readonly Microsoft.Extensions.Logging.ILogger _logger;

    public MyService(ILogger<MyService> logger)
        => _logger = logger;

    public void Run()
    {
        _logger.LogError("From injected logger");
        Serilog.Log.Logger.Error("From static logger");
    }
}

Answer 2

If you refer to the Provided Sinks list and examine the source code for some of them, you'll notice that the pattern is usually:

1. Construct the sink configuration (usually taking values from IConfiguration, inline or a combination of both)
2. Pass the configuration to the sink registration.

Then the sink implementation instantiates the required services to push logs to.

An alternate approach I could suggest is registering Serilog without any arguments (UseSerilog()) and then configure the static Serilog.Log class using the built IServiceProvider:

var host = Host.CreateDefaultBuilder(args)
    // Register your services as usual
    .UseSerilog()
    .Build()
    
Log.Logger = new LoggerConfiguration()
    .ReadFrom.Configuration(host.Services.GetRequiredService<IConfiguration>())
    .WriteTo.MessageQueueSink(host.Services.GetRequiredService<IMQProducer>())
    .CreateLogger();

host.Run();

Answer 3

A full configuration only approach is possible as long as the dependencies of your sink have a default constructor by which they will be instantiated.

In the fragment from appsettings.json below, the WriteTo section mentions the name of the Serilog activation extension method MessageQueueSink - more on that further -
and also the type of the object that must be instantiated and be passed for argument mqProducer of that extension method.

(My assembly is called MyApplication and has a MyApplication.System namespace.)

"Serilog": {
    "Using": [ "MyApplication" ],
    "MinimumLevel": {
        "Default": "Information"
    },
    "WriteTo": [
        {
            "Name": "MessageQueueSink",
            "Args": {
                "mqProducer": "MyApplication.System.MQProducer, MyApplication"
            }
        }
    ]
}

A call to ReadFrom.Configuration suffices.

namespace MyApplication.System          
{           
    class Program
    {
        static void Main(string[] args)
        {
            using var host = CreateHostBuilder(args);
            host.Services.GetService<IMyService>().Run();
            host.Run();
        }

        public static IHost CreateHostBuilder(string[] args)
            => Host.CreateDefaultBuilder(args)
                    .ConfigureServices((hostBuilderContext, services)
                        => services.AddTransient<IMyService, MyService>()
                    )
                    .UseSerilog((context, services, configuration) 
                        => configuration
                                .ReadFrom.Configuration(context.Configuration)
                    )
                    .Build();
        }
    }
}

This approach requires an extension method upon Serilog.Configuration.LoggerSinkConfiguration since this is the one mentioned in the json settings.

Since the json settings above have specified "Args": { "mqProducer": "MyApplication.System.MQProducer, MyApplication" }, an instance of that type will be passed in automatically.

public static class LoggerSinkConfigurationExtensions
{
    public static LoggerConfiguration MessageQueueSink(
        this Serilog.Configuration.LoggerSinkConfiguration loggerConfiguration,
        IMQProducer mqProducer = null
        ) =>  loggerConfiguration.Sink(new MessageQueueSink(mqProducer));        
}

If that type can not be instantiated via a default constructor you might need to do that yourself;
e.g. loggerConfiguration.Sink(new MessageQueueSink(new MQProducer( /* arguments */ )))

---

The sink

public class MessageQueueSink : ILogEventSink
{
    private readonly IMQProducer _mqProducer;

    public MessageQueueSink(IMQProducer mqProducer)
        => _mqProducer = mqProducer;
    
    public void Emit(LogEvent logEvent)
        => _mqProducer.Produce(logEvent);
}

The other services/dependencies mentioned above

public interface IMQProducer
{
    void Produce(LogEvent logEvent);
}

public class MQProducer : IMQProducer
{
    public void Produce(LogEvent logEvent)
        => Console.WriteLine($">>> Producing log event: {logEvent.RenderMessage()}");
}

public interface IMyService
{
    void Run();
}

public class MyService : IMyService
{
    private readonly Microsoft.Extensions.Logging.ILogger _logger;

    public MyService(ILogger<MyService> logger)
        => _logger = logger;

    public void Run()
    {
        _logger.LogError("From injected logger");
        Serilog.Log.Logger.Error("From static logger");
    }
}

Answer 4

@pfx help with how to load the Serilog custom sink and for this I gave him the bounty, Thanks! It was however not the final solution in my case.

Dependency Injections means that the constructor of the class will take necessary objects as parameters. While it’s not possible to load a custom sink with constructor parameters it has to be done through the service.

public static IHost CreateHostBuilder(string[] args)
           => Host.CreateDefaultBuilder(args)
                    .ConfigureServices((hostContext, services) =>
                    {
                        services
                            .AddTransient<IMyService, MyService>()
                            .AddTransient<ICommunicator, Communicator>()
                            .AddTransient<ILogEventSink, CustomSerilogSink>();
                    })
                   .UseSerilog((context, services, configuration) => configuration
                                                                       .ReadFrom.Configuration(context.Configuration)
                                                                       .ReadFrom.Services(services)
                                                                       .WriteTo.Console(LogEventLevel.Information)
                                                                       .Enrich.FromLogContext())
           .Build();
        }

The problem was that the MQ controller class had a ILogger injected and this created a circular ref resulting in frees. The MQ controller is shared so I could not change it too much, but I could make the needed methods static. Instead of using local MQ objects they was instead sent into the method.

I could however not remove the MS ILogger requirement and I still needed to be able to log within the custom sink. So first I created a Serilog like this:

private static readonly ILogger _logger = Log.ForContext<MessageQueueSink>() as ILogger;

This logger could then be wrapped in a MS Ilogger class and sent to the static producer method :

public class CustomSerilogger : Microsoft.Extensions.Logging.ILogger
{
    private readonly ILogger _logger;
    public CustomSerilogger(ILogger logger)
    { _logger = logger; }

    public IDisposable BeginScope<TState>(TState state) => default!;

    public bool IsEnabled(Microsoft.Extensions.Logging.LogLevel logLevel) { return _logger.IsEnabled(LogLevelToLogEventLevel(logLevel)); }

    public void Log<TState>(Microsoft.Extensions.Logging.LogLevel logLevel, Microsoft.Extensions.Logging.EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
    {
        if (!IsEnabled(logLevel))
            return;

        _logger.Write(LogLevelToLogEventLevel(logLevel), exception, state.ToString());
    }

    private LogEventLevel LogLevelToLogEventLevel(Microsoft.Extensions.Logging.LogLevel loglevel)
    {
        switch(loglevel)
        {
            case Microsoft.Extensions.Logging.LogLevel.Debug:
                return LogEventLevel.Debug;
            case Microsoft.Extensions.Logging.LogLevel.Information:
                return LogEventLevel.Information;
            case Microsoft.Extensions.Logging.LogLevel.Warning:
                return LogEventLevel.Warning;
            case Microsoft.Extensions.Logging.LogLevel.Error:
                return LogEventLevel.Error;
            case Microsoft.Extensions.Logging.LogLevel.Critical:
                return LogEventLevel.Fatal;
            case Microsoft.Extensions.Logging.LogLevel.None:
                return LogEventLevel.Verbose;
            case Microsoft.Extensions.Logging.LogLevel.Trace:
                return LogEventLevel.Verbose;
        }
        return LogEventLevel.Verbose;
    }
}

To configurate the sink I had to create a Custom Sink settings part in the appsettings.json file and then read this into a settings object.

var mqSinkSettings = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build()
                                                    .GetSection("MessageQueueSinkSettings").Get<MessageQueueSinkSettings>();

Finally it works!