JBoss.org Community Documentation

- Prev
- Next

## Chapter 5. Consuming Messages via Pull

There are two different ways to consume messages from a topic or queue. You can wait and have the messaging server push them to you, or you can continuously poll the server yourself to see if messages are available. This chapter discusses the latter. Consuming messages via a pull works almost identically for queues and topics with some minor, but important caveats. To start consuming you must create a consumer resource on the server that is dedicated to your client. Now, this pretty much breaks the stateless principle of REST, but after much prototyping, this is the best way to work most effectively with HornetQ through a REST interface.

You create consumer resources by doing a simple POST to the URL published by the `msg-pull-consumers` response header if you're interacting with a queue, the `msg-pull-subscribers` response header if you're interacting with a topic. These headers are provided by the main queue or topic resource discussed in Chapter 3. Doing an empty POST to one of these URLs will create a consumer resource that follows an auto-acknowledge protocol and, if you're interacting with a topic, creates a temporty subscription to the topic. If you want to use the acknowledgement protocol and/or create a durable subscription (topics only), then you must use the form parameters (`application/x-www-form-urlencoded`) described below.

autoAck

  A value of `true` or `false` can be given. This defaults to `true` if you do not pass this parameter.

durable

  A value of `true` or `false` can be given. This defaults to `false` if you do not pass this parameter. Only available on topics. This specifies whether you want a durable subscription or not. A durable subscription persists through server restart.

name

  This is the name of the durable subscription. If you do not provide this parameter, the name will be automatically generated by the server. Only usable on topics.

selector

  This is an optional JMS selector string. The HornetQ REST interface adds HTTP headers to the JMS message for REST produced messages. HTTP headers are prefixed with "http_" and every '-' charactor is converted to a '$'.

## 5.1. Auto-Acknowledge

This section focuses on the auto-acknowledge protocol for consuming messages via a pull. Here's a list of the response headers and URLs you'll be interested in.

msg-pull-consumers

  The URL of a factory resource for creating queue consumer resources. You will pull from these created resources.

msg-pull-subscriptions

  The URL of a factory resource for creating topic subscription resources. You will pull from the created resources.

msg-consume-next

  The URL you will pull the next message from. This is returned with every response.

msg-consumer

  This is a URL pointing back to the consumer or subscription resource created for the client.

### 5.1.1. Creating an Auto-Ack Consumer or Subscription

Here is an example of creating an auto-acknowledged queue pull consumer.

1. Find the pull-consumers URL by doing a HEAD or GET request to the base queue resource.

       HEAD /queues/jms.queue.bar HTTP/1.1
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       msg-create: http://example.com/queues/jms.queue.bar/create
       msg-pull-consumers: http://example.com/queues/jms.queue.bar/pull-consumers
       msg-push-consumers: http://example.com/queues/jms.queue.bar/push-consumers

2. Next do an empty POST to the URL returned in the `msg-pull-consumers` header.

       POST /queues/jms.queue.bar/pull-consumers HTTP/1.1
       Host: example.com

       --- response ---
       HTTP/1.1 201 Created
       Location: http://example.com/queues/jms.queue.bar/pull-consumers/auto-ack/333
       msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/auto-ack/333/consume-next-1

   The `Location` header points to the JMS consumer resource that was created on the server. It is good to remember this URL, although, as you'll see later, it is transmitted with each response just to remind you.

Creating an auto-acknowledged consumer for a topic is pretty much the same. Here's an example of creating a durable auto-acknowledged topic pull subscription.

1. Find the `pull-subscriptions` URL by doing a HEAD or GET request to the base topic resource

       HEAD /topics/jms.topic.bar HTTP/1.1
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       msg-create: http://example.com/topics/jms.topic.foo/create
       msg-pull-subscriptions: http://example.com/topics/jms.topic.foo/pull-subscriptions
       msg-push-subscriptions: http://example.com/topics/jms.topic.foo/push-subscriptions

2. Next do a POST to the URL returned in the `msg-pull-subscriptions` header passing in a `true` value for the `durable` form parameter.

       POST /topics/jms.topic.foo/pull-subscriptions HTTP/1.1
       Host: example.com
       Content-Type: application/x-www-form-urlencoded

       durable=true

       --- Response ---
       HTTP/1.1 201 Created
       Location: http://example.com/topics/jms.topic.foo/pull-subscriptions/auto-ack/222
       msg-consume-next: http://example.com/topics/jms.topic.foo/pull-subscriptions/auto-ack/222/consume-next-1

   The `Location` header points to the JMS subscription resource that was created on the server. It is good to remember this URL, although, as you'll see later, it is transmitted with each response just to remind you.

### 5.1.2. Consuming Messages

After you have created a consumer resource, you are ready to start pulling messages from the server. Notice that when you created the consumer for either the queue or topic, the response contained a `msg-consume-next` response header. POST to the URL contained within this header to consume the next message in the queue or topic subscription. A successful POST causes the server to extract a message from the queue or topic subscription, acknowledge it, and return it to the consuming client. If there are no messages in the queue or topic subscription, a 503 (Service Unavailable) HTTP code is returned.

## Warning

For both successful and unsuccessful posts to the msg-consume-next URL, the response will contain a new msg-consume-next header. You must ALWAYS use this new URL returned within the new msg-consume-next header to consume new messages.

Here's an example of pulling multiple messages from the consumer resource.

1. Do a POST on the msg-consume-next URL that was returned with the consumer or subscription resource discussed earlier.

       POST /queues/jms.queue.bar/pull-consumers/consume-next-1
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       Content-Type: application/xml
       msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-2
       msg-consumer: http://example.com/queues/jms.queue.bar/pull-consumers/333

       <order>...</order>

   The POST returns the message consumed from the queue. It also returns a new msg-consume-next link. Use this new link to get the next message. Notice also a msg-consumer response header is returned. This is a URL that points back to the consumer or subscription resource. You will need that to clean up your connection after you are finished using the queue or topic.

2. The POST returns the message consumed from the queue. It also returns a new msg-consume-next link. Use this new link to get the next message.

       POST /queues/jms.queue.bar/pull-consumers/consume-next-2
       Host: example.com

       --- Response ---
       Http/1.1 503 Service Unavailable
       Retry-After: 5
       msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-2

   In this case, there are no messages in the queue, so we get a 503 response back. As per the HTTP 1.1 spec, a 503 response may return a Retry-After head specifying the time in seconds that you should retry a post. Also notice, that another new msg-consume-next URL is present. Although it probabley is the same URL you used last post, get in the habit of using URLs returned in response headers as future versions of HornetQ REST might be redirecting you or adding additional data to the URL after timeouts like this.

3. POST to the URL within the last `msg-consume-next` to get the next message.

       POST /queues/jms.queue.bar/pull-consumers/consume-next-2
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       Content-Type: application/xml
       msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-3

       <order>...</order>

### 5.1.3. Recovering From Network Failures

If you experience a network failure and do not know if your post to a msg-consume-next URL was successful or not, just re-do your POST. A POST to a msg-consume-next URL is idempotent, meaning that it will return the same result if you execute on any one msg-consume-next URL more than once. Behind the scenes, the consumer resource caches the last consumed message so that if there is a message failure and you do a re-post, the cached last message will be returned (along with a new msg-consume-next URL). This is the reason why the protocol always requires you to use the next new msg-consume-next URL returned with each response. Information about what state the client is in is embedded within the actual URL.

### 5.1.4. Recovering From Client or Server Crashes

If the server crashes and you do a POST to the msg-consume-next URL, the server will return a 412 (Preconditions Failed) response code. This is telling you that the URL you are using is out of sync with the server. The response will contain a new msg-consume-next header to invoke on.

If the client crashes there are multiple ways you can recover. If you have remembered the last msg-consume-next link, you can just re-POST to it. If you have remembered the consumer resource URL, you can do a GET or HEAD request to obtain a new msg-consume-next URL. If you have created a topic subscription using the name parameter discussed earlier, you can re-create the consumer. Re-creation will return a msg-consume-next URL you can use. If you cannot do any of these things, you will have to create a new consumer.

The problem with the auto-acknowledge protocol is that if the client or server crashes, it is possible for you to skip messages. The scenario would happen if the server crashes after auto-acknowledging a message and before the client receives the message. If you want more reliable messaging, then you must use the acknowledgement protocol.

## 5.2. Manual Acknowledgement

The manual acknowledgement protocol is similar to the auto-ack protocol except there is an additional round trip to the server to tell it that you have received the message and that the server can internally ack the message. Here is a list of the respone headers you will be interested in.

msg-pull-consumers

  The URL of a factory resource for creating queue consumer resources. You will pull from these created resources

msg-pull-subscriptions

  The URL of a factory resource for creating topic subscription resources. You will pull from the created resources.

msg-acknowledge-next

  URL used to obtain the next message in the queue or topic subscription. It does not acknowledge the message though.

msg-acknowledgement

  URL used to acknowledge a message.

msg-consumer

  This is a URL pointing back to the consumer or subscription resource created for the client.

### 5.2.1. Creating manually-acknowledged consumers or subscriptions

Here is an example of creating an auto-acknowledged queue pull consumer.

1. Find the pull-consumers URL by doing a HEAD or GET request to the base queue resource.

       HEAD /queues/jms.queue.bar HTTP/1.1
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       msg-create: http://example.com/queues/jms.queue.bar/create
       msg-pull-consumers: http://example.com/queues/jms.queue.bar/pull-consumers
       msg-push-consumers: http://example.com/queues/jms.queue.bar/push-consumers

2. Next do a POST to the URL returned in the `msg-pull-consumers` header passing in a `false` value to the `autoAck` form parameter .

       POST /queues/jms.queue.bar/pull-consumers HTTP/1.1
       Host: example.com
       Content-Type: application/x-www-form-urlencoded

       autoAck=false

       --- response ---
       HTTP/1.1 201 Created
       Location: http://example.com/queues/jms.queue.bar/pull-consumers/acknowledged/333
       msg-acknowledge-next: http://example.com/queues/jms.queue.bar/pull-consumers/acknowledged/333/acknowledge-next-1

   The `Location` header points to the JMS consumer resource that was created on the server. It is good to remember this URL, although, as you'll see later, it is transmitted with each response just to remind you.

Creating an manually-acknowledged consumer for a topic is pretty much the same. Here's an example of creating a durable manually-acknowledged topic pull subscription.

1. Find the `pull-subscriptions` URL by doing a HEAD or GET request to the base topic resource

       HEAD /topics/jms.topic.bar HTTP/1.1
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       msg-create: http://example.com/topics/jms.topic.foo/create
       msg-pull-subscriptions: http://example.com/topics/jms.topic.foo/pull-subscriptions
       msg-push-subscriptions: http://example.com/topics/jms.topic.foo/push-subscriptions

2. Next do a POST to the URL returned in the `msg-pull-subscriptions` header passing in a `true` value for the `durable` form parameter and a `false` value to the `autoAck` form parameter.

       POST /topics/jms.topic.foo/pull-subscriptions HTTP/1.1
       Host: example.com
       Content-Type: application/x-www-form-urlencoded

       durable=true&autoAck=false

       --- Response ---
       HTTP/1.1 201 Created
       Location: http://example.com/topics/jms.topic.foo/pull-subscriptions/acknowledged/222
       msg-acknowledge-next: http://example.com/topics/jms.topic.foo/pull-subscriptions/acknowledged/222/consume-next-1

   The `Location` header points to the JMS subscription resource that was created on the server. It is good to remember this URL, although, as you'll see later, it is transmitted with each response just to remind you.

### 5.2.2. Consuming and Acknowledging a Message

After you have created a consumer resource, you are ready to start pulling messages from the server. Notice that when you created the consumer for either the queue or topic, the response contained a `msg-acknowledge-next` response header. POST to the URL contained within this header to consume the next message in the queue or topic subscription. If there are no messages in the queue or topic subscription, a 503 (Service Unavailable) HTTP code is returned. A successful POST causes the server to extract a message from the queue or topic subscription and return it to the consuming client. It does not acknowledge the message though. The response will contain the `acknowledgement` header which you will use to acknowledge the message.

Here's an example of pulling multiple messages from the consumer resource.

1. Do a POST on the msg-acknowledge-next URL that was returned with the consumer or subscription resource discussed earlier.

       POST /queues/jms.queue.bar/pull-consumers/consume-next-1
       Host: example.com

       --- Response ---
       HTTP/1.1 200 Ok
       Content-Type: application/xml
       msg-acknowledgement: http://example.com/queues/jms.queue.bar/pull-consumers/333/acknowledgement/2
       msg-consumer: http://example.com/queues/jms.queue.bar/pull-consumers/333

       <order>...</order>

   The POST returns the message consumed from the queue. It also returns a `msg-acknowledgemen`t link. You will use this new link to acknowledge the message. Notice also a `msg-consumer` response header is returned. This is a URL that points back to the consumer or subscription resource. You will need that to clean up your connection after you are finished using the queue or topic.

2. Acknowledge or unacknowledge the message by doing a POST to the URL contained in the `msg-acknowledgement` header. You must pass an `acknowledge` form parameter set to `true` or `false` depending on whether you want to acknowledge or unacknowledge the message on the server.

       POST /queues/jms.queue.bar/pull-consumers/acknowledgement/2
       Host: example.com
       Content-Type: application/x-www-form-urlencoded

       acknowledge=true

       --- Response ---
       Http/1.1 200 Ok
       msg-acknowledge-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/acknowledge-next-2

   Whether you acknowledge or unacknowledge the message, the response will contain a new msg-acknowledge-next header that you must use to obtain the next message.

### 5.2.3. Recovering From Network Failures

If you experience a network failure and do not know if your post to a `msg-acknowledge-next` or `msg-acknowledgement` URL was successful or not, just re-do your POST. A POST to one of these URLs is idempotent, meaning that it will return the same result if you re-post. Behind the scenes, the consumer resource keeps track of its current state. If the last action was a call to `msg-acknowledge-next`, it will have the last message cached, so that if a re-post is done, it will return the message again. Same goes with re-posting to `msg-acknowledgement`. The server remembers its last state and will return the same results. If you look at the URLs you'll see that they contain information about the expected current state of the server. This is how the server knows what the client is expecting.

### 5.2.4. Recovering From Client or Server Crashes

If the server crashes and while you are doing a POST to the `msg-acknowledge-next` URL, just re-post. Everything should reconnect all right. On the other hand, if the server crashes while you are doing a POST to `msg-acknowledgement`, the server will return a 412 (Preconditions Failed) response code. This is telling you that the URL you are using is out of sync with the server and the message you are acknowledging was probably re-enqueued. The response will contain a new `msg-acknowledge-next` header to invoke on.

As long as you have "bookmarked" the consumer resource URL (returned from `Location` header on a create, or the `msg-consumer` header), you can recover from client crashes by doing a GET or HEAD request on the consumer resource to obtain what state you are in. If the consumer resource is expecting you to acknowledge a message, it will return a `msg-acknowledgement` header in the response. If the consumer resource is expecting you to pull for the next message, the `msg-acknowledge-next` header will be in the response. With manual acknowledgement you are pretty much guaranteed to avoid skipped messages. For topic subscriptions that were created with a name parameter, you do not have to "bookmark" the returned URL. Instead, you can re-create the consumer resource with the same exact name. The response will contain the same information as if you did a GET or HEAD request on the consumer resource.

## 5.3. Blocking Pulls with Accept-Wait

Unless your queue or topic has a high rate of message flowing though it, if you use the pull protocol, you're going to be receiving a lot of 503 responses as you continuously pull the server for new messages. To alleviate this problem, the HornetQ REST interface provides the `Accept-Wait` header. This is a generic HTTP request header that is a hint to the server for how long the client is willing to wait for a response from the server. The value of this header is the time in seconds the client is willing to block for. You would send this request header with your pull requests. Here's an example:

    POST /queues/jms.queue.bar/pull-consumers/consume-next-2
    Host: example.com
    Accept-Wait: 30

    --- Response ---
    HTTP/1.1 200 Ok
    Content-Type: application/xml
    msg-consume-next: http://example.com/queues/jms.queue.bar/pull-consumers/333/consume-next-3

    <order>...</order>

In this example, we're posting to a msg-consume-next URL and telling the server that we would be willing to block for 30 seconds.

## 5.4. Clean Up Your Consumers!

When the client is done with its consumer or topic subscription it should do an HTTP DELETE call on the consumer URL passed back from the Location header or the msg-consumer response header. The server will time out a consumer with the value configured from Chapter 2.3, so you don't have to clean up if you dont' want to, but if you are a good kid, you will clean up your messes. A consumer timeout for durable subscriptions will not delete the underlying durable JMS subscription though, only the server-side consumer resource (and underlying JMS session).

- PrevChapter 4. Posting Messages
- Top of page
- Front page
- NextChapter 6. Pushing Messages