Skip to content

Commit 4804bd9

Browse files
refs #230: Clean up example projects and documentation (#233)
1 parent 7a85089 commit 4804bd9

File tree

34 files changed

+490
-541
lines changed

34 files changed

+490
-541
lines changed

README.md

Lines changed: 75 additions & 69 deletions
Large diffs are not rendered by default.

build.gradle.kts

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,7 @@ subprojects {
150150
violationRules {
151151
rule {
152152
excludes = listOf(
153-
"com.jashmore.sqs.examples*",
154-
"com.jashmore.sqs.extensions.xray.client"
153+
"com.jashmore.sqs.examples*"
155154
)
156155
element = "PACKAGE"
157156
limit {

core/README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,15 @@
11
# Java Dynamic SQS Listener Core
22

3-
This is the core implementation of the Dynamic SQS Listener API. This contains basic implementations that should cover a
4-
significant amount of use cases but if it doesn't the consumer can easily implement their own.
3+
This is the core implementation of the Dynamic SQS Listener API. This contains basic implementations that should cover a significant amount of use cases
4+
but if it doesn't, the consumer can easily implement their own.
55

66
## Usage
77

8-
See [java-dynamic-sqs-listener-core-examples](../examples/core-examples) for examples of
9-
using this core code to listen to a queue.
8+
See the [core-example](../examples/core-example) for examples of using this core code to listen to a queue in a Java application. If it is a Kotlin application,
9+
the [core-kotlin-example](../examples/core-kotlin-example) uses a [Core Kotlin DSL](../extensions/core-kotlin-dsl) for constructing a core message listener.
1010

1111
## More Information
1212

13-
For more information you can look at the root project [README.md](../README.md) which provides more information about
14-
the architecture of the application. The [API](../api) is also a good location to find more
15-
information about what each part of the framework is how they interact with each other.
13+
For more information you can look at the root project [README.md](../README.md) which provides more information about the architecture
14+
of the application. The [API](../api) is also a good location to find more information about what each part of the framework is how
15+
they interact with each other.

doc/core-implementations-overview.md

Lines changed: 32 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -12,20 +12,20 @@ The following is the diagram for how a single message would be processed through
1212
### Message Retriever
1313

1414
The [MessageRetriever](../api/src/main/java/com/jashmore/sqs/retriever) has a simple API in that it only exposes methods to obtain
15-
a message from the queue and it makes no requirements on how it should get these messages from the SQS queue. This allows for ability to optimise the
15+
a message from the queue, and it makes no requirements on how it should get these messages from the SQS queue. This allows for ability to optimise the
1616
retrieval of messages such as batching requests for retrieving messages or pre-fetching messages.
1717

1818
Core implementations include:
1919

2020
- [PrefetchingMessageRetriever](../core/src/main/java/com/jashmore/sqs/retriever/prefetch/PrefetchingMessageRetriever.java):
21-
this will prefetch messages from the queue so that new messages can be processed as soon as possible. This implementation is not appropriate if the
22-
prefetched message's visibility timeout expires before it can be picked up for process due to the message processing of previous messages taking too long.
23-
The result is that if the message has a re-drive policy it will be placed back into the queue and processed multiple times. This implementation is appropriate
21+
this will prefetch messages from the queue so that new messages can be processed as soon as possible. This implementation is not appropriate if the time
22+
to process messages is long enough for the prefetched message's visibility timeout to expire before it can be processed. In this scenario, the message's
23+
re-drive policy may place the message back into the queue resulting in it being processed multiple times. This implementation is appropriate
2424
for high volumes of messages that take little time to process.
2525
- [BatchingMessageRetriever](../core/src/main/java/com/jashmore/sqs/retriever/batching/BatchingMessageRetriever.java):
26-
This will batch requests for messages from the consumer into a single call out to the SQS queue once a certain threshold of messages were requested or at
27-
a given period if this threshold is not reached. This reduces the number of calls out to the SQS queue but reduces the performance
28-
as messages are not being requested while the batch is waiting to be built.
26+
This will batch requests for messages from the consumer into a single call out to the SQS queue once reaching the threshold of messages, or the batching
27+
timeout expires. This reduces the number of calls out to the SQS queue but can reduce the performance as no messages processing will occur while waiting
28+
for the batch size to be reached.
2929

3030
### Message Processor
3131

@@ -39,12 +39,15 @@ Core implementations include:
3939

4040
- [CoreMessageProcessor](../core/src/main/java/com/jashmore/sqs/processor/CoreMessageProcessor.java):
4141
default implementation that calls out to a `ArgumentResolverService` to resolve the arguments and calls the method.
42+
- [DecoratingMessageProcessor](../core/src/main/java/com/jashmore/sqs/processor/DecoratingMessageProcessor.java): implementation that allows for the
43+
message processing to be decorated with [MessageProcessingDecorator](../api/src/main/java/com/jashmore/sqs/decorator/MessageProcessingDecorator.java) logic.
44+
This can be useful for adding tracing, metrics or other extra functionality in the message processing.
4245

4346
### ArgumentResolverService
4447

4548
The [ArgumentResolverService](../api/src/main/java/com/jashmore/sqs/argument/ArgumentResolverService.java) is used to obtain the
4649
[ArgumentResolver](../api/src/main/java/com/jashmore/sqs/argument/ArgumentResolver.java) that can be used to populate an argument
47-
in a method when a message is being processed. For example, a parameter with the
50+
in a method when processing a message. For example, a parameter with the
4851
[@Payload](../core/src/main/java/com/jashmore/sqs/argument/payload/Payload.java) annotation will be resolved with the body
4952
of the message cast to that type.
5053

@@ -55,7 +58,7 @@ this implementation delegates to specific [ArgumentResolver](../api/src/main/jav
5558
that have been passed in. See below for the core
5659
[ArgumentResolver](../api/src/main/java/com/jashmore/sqs/argument/ArgumentResolver.java)s that are available.
5760
- [CoreArgumentResolverService](../core/src/main/java/com/jashmore/sqs/argument/CoreArgumentResolverService.java): this is
58-
a helper implementation that uses the DelegatingArgumentResolverService under the hood with all of the core
61+
a helper implementation that uses the DelegatingArgumentResolverService under the hood with the core
5962
[ArgumentResolver](../api/src/main/java/com/jashmore/sqs/argument/ArgumentResolver.java)s.
6063

6164
The core arguments that be resolved include:
@@ -64,56 +67,58 @@ The core arguments that be resolved include:
6467
this argument. This is useful if you need to forward this message to other services or want to manually extract information from the service. This is
6568
provided by the [MessageArgumentResolver](../core/src/main/java/com/jashmore/sqs/argument/message/MessageArgumentResolver.java).
6669
- [@Payload](../core/src/main/java/com/jashmore/sqs/argument/payload/Payload.java): arguments annotated with this will parse the
67-
message body into that object. If this is a String a direct transfer of the message contents is passed in, otherwise if it is a Java Bean, an attempt to
70+
message body into that object. If this is a String, the raw message body will be provided, otherwise if it is a Java Bean, an attempt to
6871
cast the message body to that bean will be used. This is provided by the
69-
[PayloadArgumentResolver](../core/src/main/java/com/jashmore/sqs/argument/payload/PayloadArgumentResolver.java), which uses
72+
[PayloadArgumentResolver](../core/src/main/java/com/jashmore/sqs/argument/payload/PayloadArgumentResolver.java) which uses
7073
a [PayloadMapper](../core/src/main/java/com/jashmore/sqs/argument/payload/mapper/PayloadMapper.java), such as
71-
the [JacksonPayloadMapper](../core/src/main/java/com/jashmore/sqs/argument/payload/mapper/JacksonPayloadMapper.java)
72-
that uses a Jackson `ObjectMapper` to parse the message body.
74+
the [JacksonPayloadMapper](../core/src/main/java/com/jashmore/sqs/argument/payload/mapper/JacksonPayloadMapper.java), to parse the message body.
7375
- [@MessageId](../core/src/main/java/com/jashmore/sqs/argument/messageid/MessageId.java): string arguments annotated with this will
7476
place the message ID of the message into this argument. This is provided by the
7577
[MessageIdArgumentResolver](../core/src/main/java/com/jashmore/sqs/argument/messageid/MessageIdArgumentResolver.java).
7678
- [Acknowledge](../api/src/main/java/com/jashmore/sqs/processor/argument/Acknowledge.java): arguments of this type will be injected
77-
with an implementation that allows for a message to be manually acknowledged when it is successfully processed. Note that if this is included in the messages
78-
signature, the [MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java) is not required to
79-
acknowledge the message after a successful execution. These implementations should be provided by the
80-
[MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java) being used.
81-
- [MessageAttribute](../core/src/main/java/com/jashmore/sqs/argument/attribute/MessageAttribute.java): arguments annotated with this
79+
with an implementation that allows for a message to be manually acknowledged when it is successfully processed. Note that if this is included,
80+
the [MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java) is not required to
81+
acknowledge the message after a successful execution and the consumer must acknowledge the message them self. The implementation of the
82+
[Acknowledge](../api/src/main/java/com/jashmore/sqs/processor/argument/Acknowledge.java) should be provided by the
83+
[MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java) instead of
84+
an [ArgumentResolver](../api/src/main/java/com/jashmore/sqs/argument/ArgumentResolver.java).
85+
- [@MessageAttribute](../core/src/main/java/com/jashmore/sqs/argument/attribute/MessageAttribute.java): arguments annotated with this
8286
will attempt to parse the contents of the message attribute into this field. For example, if the argument is a String then the attribute will be cast to a
8387
string where as if the argument is an integer it will try and parse the string into the number. This also works with POJOs in that the resolver will
8488
attempt to deserialised the message attribute into this POJO shape, e.g. via the Jackson Object Mapper. This is provided by the
8589
[MessageAttributeArgumentResolver](../core/src/main/java/com/jashmore/sqs/argument/attribute/MessageAttributeArgumentResolver.java).
86-
- [MessageSystemAttribute](../core/src/main/java/com/jashmore/sqs/argument/attribute/MessageAttribute.java); arguments annotated
90+
- [@MessageSystemAttribute](../core/src/main/java/com/jashmore/sqs/argument/attribute/MessageAttribute.java): arguments annotated
8791
with this will attempt to parse the contents of a system message attribute into this field. For example, the `SENT_TIMESTAMP` of the message can be obtained
8892
by this annotation. This is provided by the
8993
[MessageSystemAttributeArgumentResolver](../core/src/main/java/com/jashmore/sqs/argument/attribute/MessageSystemAttributeArgumentResolver.java).
9094
- [VisibilityExtender](../api/src/main/java/com/jashmore/sqs/processor/argument/VisibilityExtender.java): arguments of this type
9195
will be injected with an implementation that extends the message visibility of the current message. These implementations should be provided by the
92-
[MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java) being used.
96+
[MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java) instead of
97+
an [ArgumentResolver](../api/src/main/java/com/jashmore/sqs/argument/ArgumentResolver.java).
9398

9499
### Message Broker
95100

96101
The [MessageBroker](../api/src/main/java/com/jashmore/sqs/broker/MessageBroker.java) is the main container that controls the
97102
whole flow of messages from the [MessageRetriever](../api/src/main/java/com/jashmore/sqs/retriever) to the
98103
[MessageProcessor](../api/src/main/java/com/jashmore/sqs/processor/MessageProcessor.java). It can provide logic like the rate
99-
of concurrency of the messages being processed or when messages should be processed.
104+
of concurrency of messages processing or when messages should be processed.
100105

101106
Core implementation include:
102107

103108
- [ConcurrentMessageBroker](../core/src/main/java/com/jashmore/sqs/broker/concurrent/ConcurrentMessageBroker.java): this
104-
implementation will run on multiple threads each processing messages. It has dynamic configuration and this allows the rate of concurrency to change
105-
dynamically while the application is running.
109+
implementation will run on multiple threads each processing messages. It allows the configuration to be changed dynamically, such as changing the rate of
110+
concurrency to change while the application is running.
106111

107112
### Message Resolver
108113

109114
The [MessageResolver](../api/src/main/java/com/jashmore/sqs/resolver/MessageResolver.java) is used when the message has been
110-
successfully processed and it is needed to be removed from the SQS queue so it isn't processed again.
115+
successfully processed, and it needs to be removed from the SQS queue.
111116

112117
Core implementation include:
113118

114119
- [BatchingMessageResolver](../core/src/main/java/com/jashmore/sqs/resolver/batching/BatchingMessageResolver.java): this
115120
implementation will batch calls to delete messages from the SQS queue into a batch that will go out together once asynchronously. This is useful if you
116-
are processing many messages at the same time and it is desirable to reduce the number of calls out to SQS. A disadvantage is that the message may
117-
sit in the batch for enough time that the visibility expires and it is placed onto the queue. To mitigate this, smaller batch
118-
timeout should be used or by increasing the visibility timeout. Note you can configure this to always delete a message as soon as it is finished by
121+
are processing many messages at the same time, and it is desirable to reduce the number of calls out to SQS. A disadvantage is that the message may
122+
sit in the batch for enough time for the visibility timeout to expire, and it is placed onto the queue again. To mitigate this, a smaller batch
123+
timeout should be used or by increasing the visibility timeout. Note that you can configure this to always delete a message as soon as it is finished by
119124
setting the batch size of 1.

doc/how-to-guides/core/core-how-to-add-brave-tracing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ the [DecoratingMessageProcessor](../../../core/src/main/java/com/jashmore/sqs/pr
2626

2727
## Example
2828

29-
See the [core-examples](../../../examples/core-examples) for a basic application that is running with tracing enabled automatically.
29+
See the [core-example](../../../examples/core-example) for a basic application that is running with tracing enabled automatically.
3030

3131
## Including Brave information in your SQS messages
3232

doc/how-to-guides/core/core-how-to-implement-a-custom-message-retrieval.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ is written in a non-blocking programming style. The current core implementation
1212
[retriever package](../../../core/src/main/java/com/jashmore/sqs/retriever).
1313

1414
Once you have built your own retriever you can see how this can be integrated into the framework by looking in the [examples](../../../examples) directory
15-
and more specifically in the [core-examples module](../../../examples/core-examples).
15+
and more specifically in the [core-example](../../../examples/core-example) module.
1616

1717
### Integrating the new message retriever into the spring app
1818

doc/how-to-guides/how-to-connect-to-aws-sqs-queue.md

Lines changed: 2 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -10,26 +10,6 @@ In this example we are defining the SQS Properties needed via Environment variab
1010
documentation about different ways to define these properties. The [Spring AWS Example](../../examples/spring-aws-example) will be
1111
used for this guide.
1212

13-
## Steps
13+
## Usage
1414

15-
1. Create a new AWS account. You can Google for links on where to do this.
16-
1. Create a new SQS Queue in a region near you, e.g. us-east-2 (Ohio). You will need the region and Queue URL.
17-
1. Create a new IAM user that has full permission (for simplicity, otherwise you can put a more restricted permissions) to this SQS queue. You will
18-
need the Access Key ID and Secret Access Key.
19-
1. Change directory to the AWS Spring example.
20-
21-
```bash
22-
cd examples/java-dynamic-sqs-listener-spring-aws-example
23-
```
24-
25-
1. Run the Spring Boot application with the AWS details recorded above. For example:
26-
27-
```bash
28-
AWS_ACCESS_KEY_ID={KEY_RECORDED_ABOVE} \
29-
AWS_REGION={REGION_QUEUE_CREATED_IN_ABOVE} \
30-
AWS_SECRET_ACCESS_KEY={SECRET_KEY_RECORDED_ABOVE} \
31-
SQS_QUEUE_URL={FULL_URL_OF_SQS_QUEUE} \
32-
gradle bootRun
33-
```
34-
35-
1. Send a message to the Queue by right clicking the queue in the AWS Console and selecting Send Message
15+
See the [Spring AWS Example README.md](../../examples/spring-aws-example/README.md) for steps to use this yourself.
-1.26 KB
Loading

0 commit comments

Comments
 (0)