I have a Play Framework 2.0 Java application hosted on Heroku, and I am monitoring it using the free-tier New Relic addon. For most of the transactions, a majority of the time is spent in what New Relic labels as Actor[akka:\\play\deadLetters].tell(). What is the application actually doing during this time?
As a simple description, Akka (http://en.wikipedia.org/wiki/Akka_(toolkit); http://akka.io/) is part of the Play framework as one of their integrations. As the application on Play is instrumented for monitoring there are HTTP requests made by Akka that are traced as a web transaction. In short, we measure it. As for what is is specifically doing, I recommend checking the Play documentation or the Akka link from the first sentence.
If you have a Java agent version older than 3.2.0, upgrading the Java agent will give you the following change:
akka.actor.ActorKilledException is now ignored by default
The ActorKilledException is commonly thrown in Play applications as a control mechanism in normally functioning applications. In previous versions, this exception inflated the reported error rate. These exceptions are now ingored by default. You can override the default ignore_errors list to provide your own exceptions or to omit the ActorKilledException.
Let us know if this information is helpful or if you need additional assistance.
Jeanie Swan New Relic Support
I'm not very familiar with how NewRelic collects data, however deadLetters is a special Actor that receives "all messages that were sent to a dead (or non existent) Actor". You can read more about dead letters in the official docs.
For example you can subscribe to these dead letters and print them (which should give you enough information to then track down their source and fix it). A typical case where many dead letters may be encountered is when you're sending messages to an Actor which has stopped, but someone is still sending messages to it - this you should be able to detect once printing the deadletters.
