Polly retry unit test

Question

I am using polly to handle retry (see below code). How can I unit test polly retry? using xunit and moq

services.AddHttpClient("GitHub", client =>
{
    client.BaseAddress = new Uri("https://api.github.com/");
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
})
.AddTransientHttpErrorPolicy(builder => builder.WaitAndRetryAsync(new[]
{
    TimeSpan.FromSeconds(1),
    TimeSpan.FromSeconds(5),
    TimeSpan.FromSeconds(10)
}));

I recommend looking up a library called [simmy](https://github.com/Polly-Contrib/Simmy) which allows you to set up chaos policies. These policies will allow you to set a percentage of how often a http request will cause a transient error. — Hayden, Jul 21 '22 at 12:24
Does this answer your question? [retry polly unit testing (xunit and moq)](https://stackoverflow.com/questions/73087890/retry-polly-unit-testing-xunit-and-moq) — Peter Csala, Jul 24 '22 at 16:05
As I stated in [this answer](https://stackoverflow.com/a/73096084/13268855) you can't unit test such code, since the retry policy is attached to the HttpClient via the DI. In case of unit testing you are not relying on your DI. This integration can be tested via an integration or component test. — Peter Csala, Jul 24 '22 at 16:07
Hi, There is a nice way to test these type of scenario using Http interceptions - using JustEat nuget, checkthis out -> https://github.com/justeat/httpclient-interception if you polly retry policy is in a an extension method then add that policy to test code and you can assert it — Shankar S, Mar 13 '23 at 09:22

score 1 · Answer 1 · edited Jul 27 '22 at 08:34

As suggested in the comments I recommend Simmy.

It allows you to inject exceptions, return BadRequests and etc. in order to trigger Polly's fault and resilience policies such as WaitAndRetry.

These are a few samples from the documentation.

Inject (Socket) exception

var chaosPolicy = MonkeyPolicy.InjectException(Action<InjectOutcomeOptions<Exception>>);

For example: it causes the policy to throw SocketException with a probability of 5% if enabled

var fault = new SocketException(errorCode: 10013);
var chaosPolicy = MonkeyPolicy.InjectException(with =>
    with.Fault(fault)
        .InjectionRate(0.05)
        .Enabled()
    );

Inject (BadRequest) result

var chaosPolicy = MonkeyPolicy.InjectResult(Action<InjectOutcomeOptions<TResult>>);

For example: it causes the policy to return a bad request HttpResponseMessage with a probability of 5% if enabled

var result = new HttpResponseMessage(HttpStatusCode.BadRequest);
var chaosPolicy = MonkeyPolicy.InjectResult<HttpResponseMessage>(with =>
    with.Result(result)
        .InjectionRate(0.05)
        .Enabled()
);

Simply set the InjectionRate to 1 to guarantee a fault in your unit test.

If you want to use the InjectionRate less than 1 you can use xunit and moq chaining via SetupSequence and Moq.Language.ISetupSequentialResult. Here's an example from an blockchain challenge I had to do, I execute 4 calls in a row, so if the InjectionRate is 0.25 one of the 4 calls would trigger a Polly policy:

[Fact]
public async Task Should_Return_GetEthereumTransactionsAsync()
{
    // Arrange
    IConfiguration settings = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();
    IOptions<Settings> optionSettings = Options.Create(new Settings
    {
        CompanyKeyAPI = settings.GetSection("CompanyKeyAPI").Value,
        ProjectId = settings.GetSection("ProjectId").Value
    });

    var sequenceHttpResponse = new List<Tuple<HttpStatusCode, HttpContent>>
    {
        new Tuple<HttpStatusCode, HttpContent>(HttpStatusCode.OK, ApiCompanyKeyResponses.EthereumBlockWithTransactionHashs()),
        new Tuple<HttpStatusCode, HttpContent>(HttpStatusCode.OK, ApiCompanyKeyResponses.Transaction(1)),
        new Tuple<HttpStatusCode, HttpContent>(HttpStatusCode.OK, ApiCompanyKeyResponses.Transaction(2)),
        new Tuple<HttpStatusCode, HttpContent>(HttpStatusCode.OK, ApiCompanyKeyResponses.Transaction(3))
    };

    IHttpClientFactory httpClientFactory = base.GetChainedCompanyKeyHttpClientFactory(new Uri(Path.Combine(optionSettings.Value.CompanyKeyAPI, optionSettings.Value.ProjectId)), sequenceHttpResponse);
    CompanyKeyService CompanyKeyService = new CompanyKeyService(httpClientFactory);

    // Act
    List<EthereumTransaction> ethereumTransactionsResult = CompanyKeyService.GetEthereumTransactionsAsync(blockNumber, address).Result;

    // Assert
    Assert.IsType<List<EthereumTransaction>>(ethereumTransactionsResult);
    Assert.NotNull(ethereumTransactionsResult);
    Assert.Equal(ethereumTransactionsResult.Count, 3);
    Assert.Equal(ethereumTransactionsResult[0].result.blockHash, blockHash);
}


public IHttpClientFactory GetChainedCompanyKeyHttpClientFactory(Uri uri, List<Tuple<HttpStatusCode, HttpContent>> httpReturns, HttpStatusCode statusCode = HttpStatusCode.OK)
{
    Mock<HttpMessageHandler> httpMsgHandler = new Mock<HttpMessageHandler>();
    var handlerPart = httpMsgHandler.Protected().SetupSequence<Task<HttpResponseMessage>>("SendAsync", new object[2]
    {
        ItExpr.IsAny<HttpRequestMessage>(),
        ItExpr.IsAny<CancellationToken>()
    });

    foreach (var httpResult in httpReturns)
    {
        handlerPart = AddReturnPart(handlerPart, httpResult.Item1, httpResult.Item2);
    }
    httpMsgHandler.Verify();

    HttpClient client = new HttpClient(httpMsgHandler.Object)
    {
        BaseAddress = uri
    };
    Mock<IHttpClientFactory> clientFactory = new Mock<IHttpClientFactory>();
    clientFactory.Setup((IHttpClientFactory cf) => cf.CreateClient(It.IsAny<string>())).Returns(client);
            
    return clientFactory.Object;
}

private Moq.Language.ISetupSequentialResult<Task<HttpResponseMessage>> AddReturnPart(Moq.Language.ISetupSequentialResult<Task<HttpResponseMessage>> handlerPart,
HttpStatusCode statusCode, HttpContent content)
    {
        return handlerPart

        // prepare the expected response of the mocked http call
        .ReturnsAsync(new HttpResponseMessage()
        {
            StatusCode = statusCode, 
            Content = content
        });
    }

....

public class CompanyKeyService : ICompanyKeyService
{
    private readonly IHttpClientFactory _clientFactory;
    private readonly HttpClient _client;
    public CompanyKeyService(IHttpClientFactory clientFactory)
    {
        _clientFactory = clientFactory;
        _client = _clientFactory.CreateClient("GitHub");
    }

    public async Task<List<EthereumTransaction>> GetEthereumTransactionsAsync(string blockNumber, string address)
    {
        //Validation removed...
   
        List<string> transactionHashs = await GetEthereumTransactionHashsByBlockNumberAsync(blockNumber);
        if (transactionHashs == null) throw new Exception("Invalid entry. Please check the Block Number.");
        var tasks = transactionHashs.Select(hash => GetTransactionByHashAsync(hash, address));
        EthereumTransaction[] lists = await Task.WhenAll(tasks);
        return lists.Where(item => item != null).ToList();
    }
}

According to my understanding in your provided sample you are making asserting only against the result. So, how does it test the integration between the HttpClient and the retry policy? — Peter Csala, Jul 27 '22 at 08:39

score -1 · Answer 2 · answered Jul 21 '22 at 12:20

You can unit test this by mocking out the HttpClient and setting up your own test version of the WaitAndRetryAsync policy. Example:

var mockHttpClient = new Mock<HttpClient>();
var mockRetryPolicy = new Mock<IAsyncPolicy<HttpResponseMessage>>();

mockRetryPolicy
    .Setup(p => p.ExecuteAsync(It.IsAny<Func<Context, CancellationToken, Task<HttpResponseMessage>>>(), It.IsAny<Context>(), It.IsAny<CancellationToken>()))
    .ReturnsAsync(new HttpResponseMessage());

var services = new ServiceCollection();
services
    .AddHttpClient("GitHub", client =>
    {
        client.BaseAddress = new Uri("https://api.github.com/");
        client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
    })
    .AddTransientHttpErrorPolicy(builder => mockRetryPolicy.Object);

var serviceProvider = services.BuildServiceProvider();
var httpClientFactory = serviceProvider.GetRequiredService<IHttpClientFactory>();
var httpClient = httpClientFactory.CreateClient("GitHub");

Assert.NotNull(httpClient);

This only tests that a mock is being called, not that the retry policy is working. — Hayden, Jul 21 '22 at 12:26
found this https://github.com/App-vNext/Polly/wiki/Unit-testing-with-Polly---with-examples — itaustralia, Jul 22 '22 at 04:33
In your test you recreate an alternative HttpClient + retry integration. So, this code does not test any part of the original code. — Peter Csala, Jul 24 '22 at 16:10

Polly retry unit test

2 Answers2

Linked