Expose S3 Request Metrics

[azkrishpy]

Issue #, if available:
#806
Description of changes:
Exposes S3 Request Metrics from the CRT S3 Client.

• S3RequestMetrics: Captures timing, request/response info, and error details for each S3 request attempt
• onTelemetry() callback: New method in S3MetaRequestResponseHandler invoked after each request completes
• ErrorType classification: Categorizes errors (SUCCESS, THROTTLING, SERVER_ERROR, CONFIGURED_TIMEOUT, IO, OTHER)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[azkrishpy]

Java SDK users can use a custom MetricPublisher that bridges CRT metrics to SDK's MetricCollection:

public class CrtMetricPublisher implements MetricPublisher {
    private final MetricPublisher delegate;

    public CrtMetricPublisher(MetricPublisher delegate) {
        this.delegate = delegate;
    }

    public void publishCrtMetrics(S3RequestMetrics crtMetrics) {
        MetricCollection collection = MetricCollection.builder()
            .name("S3Request")
            .creationTime(Instant.now())
            .putMetric(CoreMetric.API_CALL_DURATION, 
                Duration.ofNanos(crtMetrics.getApiCallDurationNs()))
            .putMetric(CoreMetric.SERVICE_ID, crtMetrics.getServiceId())
            .putMetric(CoreMetric.OPERATION_NAME, crtMetrics.getOperationName())
            .putMetric(CoreMetric.RETRY_COUNT, crtMetrics.getRetryCount())
            .putMetric(CoreMetric.AWS_REQUEST_ID, crtMetrics.getAwsRequestId())
            .putMetric(CoreMetric.AWS_EXTENDED_REQUEST_ID, crtMetrics.getAwsExtendedRequestId())
            .putMetric(CoreMetric.BACKOFF_DELAY_DURATION, 
                Duration.ofNanos(crtMetrics.getBackoffDelayDurationNs()))
            .putMetric(CoreMetric.SERVICE_CALL_DURATION, 
                Duration.ofNanos(crtMetrics.getServiceCallDurationNs()))
            .putMetric(CoreMetric.SIGNING_DURATION, 
                Duration.ofNanos(crtMetrics.getSigningDurationNs()))
            .build();

        delegate.publish(collection);
    }

    @Override
    public void publish(MetricCollection metricCollection) {
        delegate.publish(metricCollection);
    }

    @Override
    public void close() {
        delegate.close();
    }
}

Usage

CrtMetricPublisher metricPublisher = new CrtMetricPublisher(
    LoggingMetricPublisher.create());

S3MetaRequestResponseHandler responseHandler = new S3MetaRequestResponseHandler() {
    @Override
    public void onTelemetry(S3RequestMetrics metrics) {
        metricPublisher.publishCrtMetrics(metrics);
    }

    @Override
    public void onFinished(S3FinishedResponseContext context) {
        // Handle completion
    }
};

S3MetaRequestOptions options = new S3MetaRequestOptions()
    .withMetaRequestType(MetaRequestType.GET_OBJECT)
    .withHttpRequest(httpRequest)
    .withResponseHandler(responseHandler);

try (S3MetaRequest request = client.makeMetaRequest(options)) {
    // Request executes, metrics published via onTelemetry callback
}

[azkrishpy]

The following metrics are not available from CRT for java since sdk has the information:

• CredentialsFetchDuration
• EndpointResolveDuration
• MarshallingDuration
• ServiceEndpoint

[zoewangg]

What do you mean by partial execution? When is this invoked?

[azkrishpy]

When there are multiple parallelized request created for a single meta request, each request gets its own response with metrics. Although the meta request might not have been fully executed, the request might be complete and hence customers receive the metric data for that request. This is what I meant by partial execution of meta request.

[zoewangg]

Gotcha, can we update the documentation to make it more clear?

[azkrishpy]

Fixed, thanks!

[TingDaoK]

All trivial comments, look good to me as long as Java SDK team align with the interface provided!

[TingDaoK]

new line at the end of the file

[zoewangg]

Minor: is this deprecation intended? CRT.awsGetErrorType() doesn't exactly return if an error should be retryable or not though.

[azkrishpy]

Noticed that java uses isErrorRetryable to figure out if the error needs to be pushed as an IOException or not. Since isErrorRetryable(HttpException exception) was always a temporary method to convey errors and we now provide the correct error type, we are deprecating that method.

[zoewangg]

I see, we use it in Java https://github.com/aws/aws-sdk-java-v2/blob/master/http-clients/aws-crt-client/src/main/java/software/amazon/awssdk/http/crt/internal/CrtUtils.java#L47

I suppose we should change it to if(CRT.awsGetErrorType(httpException.getErrorCode()) == IO) in Java. Is there any non-I/O code that we should retry? I'd expect no, but wanted to double check

[zoewangg]

This seems useful, can we expose the getter?

[zoewangg]

What's requestPtr?

[zoewangg]

Can we update the doc to add the definition of transient error?