Custom Instrumentation in Java with New Relic using @Trace

When integrating monitoring tools like New Relic with a Java application, a developer may encounter situations where certain libraries or frameworks are not automatically instrumented.  

This is often the case with specialized libraries such as Google Cloud Platform (GCP) libraries or other third-party libraries.  

To address this limitation, New Relic provides a way to add custom instrumentation using the @Trace annotation. This allows for manual instrumentation of code to capture performance metrics and transactions of the full stack trace.

Issue: Missing instrumentation for unsupported libraries

New Relic’s automatic instrumentation is extensive, but it does not cover every possible library. For example, when using GCP libraries, an organization might find that critical parts of the application are not being monitored, resulting in gaps in performance data. This makes it difficult to diagnose issues or understand the performance characteristics of JAVA-based applications.

Solution: Custom instrumentation with @Trace

New Relic supports adding custom instrumentation to Java code using the @Trace annotation. This annotation can be applied to methods to capture detailed transaction traces and performance metrics. Here is how a developer can do it:

Step-by-Step Guide to Custom Instrumentation

1. Add New Relic dependency: Ensure that your project includes the New Relic Java agent. This is typically done by adding the New Relic agent JAR to your application’s startup class path.

2. Annotate methods with @Trace: Identify the methods in your code that you want to instrument. Apply the @Trace annotation to these methods. You can also specify additional attributes to customize the trace.

3. Deploy and monitor: Deploy your application and monitor the new custom traces in the New Relic dashboard.

The following is an example of using the @Trace annotation in a Java application using a GCP library:

import com.newrelic.api.agent.Trace; 
import com.google.cloud.storage.Storage; 
import com.google.cloud.storage.StorageOptions; 
import com.google.cloud.storage.Blob; 

public class GcsService { 

    private Storage storage; 

    public GcsService() { 
        storage = StorageOptions.getDefaultInstance().getService(); 
    } 

    @Trace(dispatcher = true) 
    public void uploadFile(String bucketName, String objectName, byte[] content) { 
        Blob blob = storage.create(Blob.newBuilder(bucketName, objectName).build(), content); 
        System.out.println("File uploaded to " + blob.getSelfLink()); 
    } 

    @Trace(dispatcher = true) 
    public byte[] downloadFile(String bucketName, String objectName) { 
        Blob blob = storage.get(bucketName, objectName); 
        return blob.getContent(); 
    } 
} 

In this example, the uploadFile and downloadFile methods are annotated with @Trace. This tells New Relic to collect transaction traces for these methods, providing visibility into their performance.

Additional customization

The @Trace annotation can be customized further with bellows attributes.

dispatcher

• Type: boolean

• Default: false

• Description: Indicates whether the method should be treated as a dispatcher transaction. Dispatcher transactions are entry points for requests, such as servlets or message consumers.

metricName

• Type: String

• Default: None

• Description: Specifies a custom metric name for the transaction trace. By default, the metric name is derived from the method signature. Custom metric names can make it easier to identify and analyze specific transactions in the New Relic dashboard.

nameTransaction

• Type: String

• Default: None

• Description: Specifies a name for the transaction. This can be useful for grouping similar transactions under a single name, providing more meaningful insights in the New Relic UI.

skipTransactionTrace

• Type: boolean

• Default: false

• Description: If set to true, the method will be included in transaction metrics but excluded from transaction traces. This can reduce overhead if detailed traces are not needed for certain methods.

metricAggregator

• Type: String

• Default: None

• Description: Specifies a custom aggregator name for the metrics. This can be useful for aggregating metrics under a specific name, especially when you want to group metrics from different methods or classes.

Here is an example of how you might use these attributes in a Java application:

‍

import com.newrelic.api.agent.Trace; 
import com.google.cloud.storage.Storage; 
import com.google.cloud.storage.StorageOptions; 
import com.google.cloud.storage.Blob; 

public class GcsService { 

    private Storage storage; 

    public GcsService() { 
        storage = StorageOptions.getDefaultInstance().getService(); 
    } 
    
    @Trace(dispatcher = true, metricName = "Custom/GcsService/uploadFile", nameTransaction = "GCSUploadTransaction", skipTransactionTrace = true) 
    public void uploadFile(String bucketName, String objectName, byte[] content) { 
        Blob blob = storage.create(Blob.newBuilder(bucketName, objectName).build(), content); 
        System.out.println("File uploaded to " + blob.getSelfLink()); 
    } 
} 

Suggestions and best practices

1. Identify critical paths: Focus on instrumenting the critical paths of your application where performance monitoring is most needed.

2. Use meaningful metric names: Customize the metric names to make it easier to identify and analyze the data in the New Relic dashboard.

3. Monitor overhead: While custom instrumentation is powerful, it can add overhead. Monitor the performance impact and use it judiciously.

4. Leverage New Relic’s documentation: New Relic provides extensive documentation and examples. Leverage these resources to get the most out of custom instrumentation.

‍

Using a custom class to do instrumentation

As New Relic Java agent jar is used to instrument JAVA code, it is not desirable to have the New Relic jar file on the class path again for custom instrumentation. In this case, the following solution will help.

Create custom trace annotation

import java.lang.annotation.ElementType; 
import java.lang.annotation.Retention; 
import java.lang.annotation.RetentionPolicy; 
import java.lang.annotation.Target; 

@Target(ElementType.METHOD) 
@Retention(RetentionPolicy.RUNTIME) 
public @interface Trace { 
    public static final String NULL = ""; 
    String metricName() default NULL; 
    boolean dispatcher() default false; 
    String tracerFactoryName() default NULL; 
} 

Update New Relic configuration

To tell New Relic to use a custom @Trace annotation, update the NewRelic.yml configuration file with the trace_annotation_class_name property.

common: &default_settings 
  ... 
  class_transformer: 
    ... 
    trace_annotation_class_name: <yourpackage>.Trace 
  ... 

Conclusion

Custom instrumentation with New Relic’s @Trace annotation is a powerful way to fill in the gaps when automatic instrumentation falls short. By manually instrumenting code, an organization can ensure comprehensive monitoring and gain valuable insights into the performance of applications, even when using New Relic unsupported libraries, such as a GCP API library.

Implementing custom instrumentation requires careful planning and monitoring, but the benefits in terms of visibility and performance management make it a valuable tool in the development and operations toolkit.