How to add metrics to external services using aioprometheus and FastAPI?

Question

I'm trying to add metrics to external services with aioprometheus in an app built with FastAPI. Here is a simplified example of what I'm trying to achieve.

Say I have a wrapper App class as such:

from aioprometheus import Registry, Counter, Histogram
from fastapi import FastAPI

class App:
    def __init__(self, ...):
        self.registry = Registry()
        self.counter = Counter(
            name="counts", doc="request counts"
        )
        self.latency = Histogram(
            name="latency",
            doc="request latency",
            buckets=[0.1, 0.5, 1, 1.5, 2]
        )

        self.app = FastAPI()
        self._metrics()

    def _metrics(self):
        # Counter metrics
        @self.app.middleware("http")
        async def counter_metrics(request, call_next):
            response = await call_next(request)
            self.counter.inc(
                {"path": str(request.url.path), "status": response.status_code}
            )
            return response

        # Latency metrics
        @self.app.middleware("http")
        async def latency_metrics(request, call_next):
            start = time.time()
            response = await call_next(request)
            total_duration = time.time() - start
            self.latency.observe(
                {"path": str(request.url.path)}, total_duration
            )
            return response
        
        @self.app.on_event("startup")
        async def startup():
            self.app.include_router(some_router(...))

        self.registry.register(self.counter)
        self.registry.register(self.latency)

Basically, I have Registry, Counter, and Histogram initiated. In _metrics, I have Counter and Histogram specific logics that are later added to Registry. This will do its magic and catch the metrics when an endpoint in some_router is called (this is good! I would want to keep this, as well as having the external service metrics).

However, say I call an external service from some_router as such:

from fastapi import APIRouter

def some_router():
    router = APIRouter()

    @router.get("/some_router")
    async def some_router():
        response = await external_service()

        return response

    return router

In this case, how would I add metrics specifically to external_service, i.e., Latency of this specific external service?

Chris · Answer 1 · 2022-10-12T07:50:09.013

As per the documentation, you would need to attach your metrics to the app instance using the generic app.state attribute (see the implementation of Starlette's State class as well), so they can easily be accessed in the route handler—as metrics are often created in a different module than where they are used (as in your case). Thus, you could use the following in your App class, after instantiating the metrics:

self.app.state.registry = registry
self.app.state.counter = counter
self.app.state.latency = latency

In your routers module, you could get the app instance using the Request object, as described here and here, and then use it to get the metrics instances (as shown below), which will let you add metrics to your external_service:

from fastapi import Request
...
@router.get("/some_router")
async def some_router(request: Request):
    registry = request.app.state.registry
    counter = request.app.state.counter
    latency = request.app.state.latency

    response = await external_service()
    
    return response

How to add metrics to external services using aioprometheus and FastAPI?

1 Answers1

Linked