Java 8 and LAMBDAS

A little bit late to the party with this one. I’ve been keeping up-to-date with Java 8 (and 9) changes, but our enterprise servers tend to not to be updated early on (understandably).

However, Oracle ran a Java 8 MOOC recently (they’ve since made the videos available via YouTube). I’m particularly interested in the parallel streams as we have a long running process that may benefit from the new syntax.

An unscientific test on my own laptop comparing the standard Java loop (an example from the MOOC):

for (Student s : students) {
   if (s.getGradYear() == 2011) {
      if (s.getScore() > highestScore) {
         highestScore = s.getScore();

Takes approximately 469 ms to run.

However, using the new Lambda syntax:

highestScore = students.parallelStream()
                  .filter(s -> s.getGradYear() == 2011)
                  .mapToDouble(s -> s.getScore())

This comes down to 272 ms.

As I say, unscientific and a very simple use case. Be interesting to see what happens in the real world though.

Oracle sql loader investigations

Today I’ve been exposed to the world of SQL Loader. There is a process that runs on a daily basis that does the following:

1. Parse a text file and copy to a remote server
2. Use SQL Loader to append the contents of this file to a database table
3. A final process to concatenate 2 fields in the database after the insert has completed.

Step 3 is an unnecessary overhead that was introduced a while back. Steps 1 or 2 are the better areas for this fix to be implemented. Having never used SQL Loader it looked a more interesting learning experience, and heres some quick findings.

To read the value into a temporary store, use the BOUNDFILLER key word (apparently supported in Oracle 9+). However, with our implementation, then continue to process the columns in order, before finally coming back to insert the temp values into their relevant columns.

An example:

The text file looks roughly like this:


And the database table has the following columns


The SQL Loader to concatenate File Entry [0] and File Entry [3] is as follows:

ID “:tempID”,
CODE “:tempCode||:tempID”

Found plenty of examples on the Internet, but few made reference to the fields needing to be the final ones inserted.

Unfinished Business–Part 1

Baldock Beast. Its a local half marathon, virtually on my door-step. The 3rd running of the event happened last weekend (15/2). And I recorded my third DNS (Did Not Start).

  • 2013 – My wife was pregnant, but it was prior to scans so no-one knew. My running buddy had just become a dad, and his son had a bout of measles. Measles and pregnant ladies don’t mix. I couldn’t think of a way to explain not wanting to run with my mate, and with a slight hint of a cold decided to pull the ‘flu flag’. DNS 1.
  • 2014 – I headed into London for a Cage The Elephant / Foals gig. By 6pm the beers and shots were flowing, and I knew I’d be in no fit state the next day. I wasn’t. I may have sobered up by the Monday. Just. DNS 2.
  • 2015 – I’ve been slightly limping for a year of so. My right knee. It hasn’t interfered with training, but 2 people this year have asked if I was injured – when I felt fine (I think I’d got use to the limp). On the Thursday before this years race I heard those dreaded words from a physio – ‘I’d advise you don’t run’. DNS 3.

I’ll be entering again next year. Lets hope I make the start line this time round.

PS. Knee is nothing too much to worry about. A tight tendon that needs loosen off, a slight alteration to my running style and perhaps a change of trainers. Latest diagnosis was for about 6-8 weeks off. In better news I can continue to cycle and swim. No Crossfit for the time being either though.

Generating SHA Mac Using Java

A team member was struggling to get this working recently, and noted that the examples he encountered on the Internet didn’t seem to be correct.

Heres the ‘working’ example that we have put in place – open to feedback if something silly is being done.

public static String generateHMac( String key, String baseString ) throws UnsupportedEncodingException, InvalidKeyException, NoSuchAlgorithmException {
SecretKeySpec secret = new SecretKeySpec(key.getBytes(), "HmacSHA1");
Mac mac = Mac.getInstance("HmacSHA1");

byte[] resultArray = mac.doFinal(baseString.getBytes("UTF-8"));
byte[] hexBytes = new Hex().encode(resultArray);
return new String(hexBytes, "UTF-8");

JMS and GlassFish 4–The Overview

As part of the latest Java EE release (version 7), the JMS component has been reviewed and revamped. I was recently fortunate enough to attend the inaugural London GlassFish Users Group earlier this week where Nigel Deakin from Oracle was presenting the latest features and developments.

This blog post is really to expand my knowledge, test some of the things mentioned, and to act as reference to my future self.

1. Getting Started

All examples in this blog post are going to be using GlassFish 4.0 and NetBeans 7.3.

1.1 Queue Creation

First of all start the GlassFish server, and then log into the Admin Console (localhost:4848). We’re then going to create our Queue and Connection Factory to handle the queue.

Next up, create the queue by navigating to Resources –> JMS Resources –> Destination Resources, and click New. Then enter the following parameters:


Then create the Connection Factory by going to Resources –> JMS Resources –> Destination Resources, and enter the following parameters:


Thats us got the infrastructure for the queue to function. (I’ve subsequently discovered there is no need for the Connection Factory, GF4 provides a default one that would be used if no others have been created!).

1.2 Sending First Message (using JMS 1.1)

Now return to NetBeans and lets create a servlet to send the first message. We’re going to use JMS 1.1 for this example, before then doing JMS 2.0 which will hopefully show some of the simplifications that have been made.

In NetBeans create a new servlet, named something like JMSOne. Then add the following code to the processRequest() method.

Connection connection = null;
ConnectionFactory connectionFactory = null;
Destination dest = null;
Context jndiContext = null;

try {
   jndiContext = new InitialContext();
   connectionFactory = (ConnectionFactory) jndiContext.lookup("jms/testQueueFactory");
   dest = (Destination) jndiContext.lookup("jms/testQueue");

   connection = connectionFactory.createConnection();

   // The 2 parameters passed here do not make any difference in GF!
   Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
   MessageProducer producer = session.createProducer(dest);
   TextMessage message = session.createTextMessage();
   message.setText("First test message!");
} catch (NamingException | JMSException e) {
   System.err.println("Exception raised: " + e.getMessage());
} finally {
   if (connection != null) {

   if (jndiContext != null) {

Run the servlet, and perhaps have some output, either to the screen or to the GlassFish log and your message should send. (How do I know its been sent? Well, this is something that I think is lacking in GF – or I’m missing something simple. I can’t see where you can see the current status of the queue – but I’ll see if I can address this later in this blog post).

1.3 Retrieving the Message

Now we need to create a message consumer to read the message that we have just sent. NetBeans makes this incredible straight-forward to do by providing most of the code for you.

Right click the project and create New->Enterprise Java Beans –> Message Driven Beans and then enter the parameters for the queue that we created previously.

The code to retrieve the bean is entered into the onMessage() method and looks like:

public void onMessage(Message message) {
   String msg;
   try {
      msg = ((TextMessage) message).getText();
      Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, msg);
   } catch (JMSException ex) {
      Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, null, ex);

1.4 Sending using JMS 2.0

So the code previous is straight-forward, but slightly long winded. Admittedly some could be removed with try with resources functionality introduced in JavaEE 7, but still. Now create a new Java Servlet, along the lines of JMSTwo.

The code to send the message now looks like this (using JMS 2.0) (note, I’m calling sendMessage() from my processRequest method):

@Resource(lookup = "jms/testQueue")
private Queue queue;
private JMSContext jmsContext;

private void sendMessage() {
   String msg = "My JMS 2.0 Message";
   jmsContext.createProducer().send(queue, msg);

The code is now down to 2 lines, and 1 of those is to declare the data to send. There is no need for the session, connection factory or connection. Everything is taken care of.

2. Other JMS 2.0 Improvements

Alongside the simplifications to sending a message, JMS 2.0 has brought about a few other enhancements. These are:

  • Delayed Delivery
  • Async Sending
  • Improvements to Poison Message Handling

2.1 Delayed Delivery Example

The code for this a very straight-forward. The time is in milliseconds, so the code below delays sending the message for 60 seconds. Note this doesn’t hold the servlet from continuing. By watching my server.log for the 60 seconds after, we see the text from the message appear.

jmsContext.createProducer().setDeliveryDelay(60000).send(queue, msg);

2.2 Async Sending

This doesn’t fit well into this blog post – its for use with standalone applications outside of the Java EE Web and EJB containers. It does however allow the application to send a message and then continue processing further steps without the need to wait for a reply.

A future blog post may look into standalone applications sending messages.

2.3 Poison Message Handling

Poison Message Handling really comes into effect when a message is returned to the queue for re-processing for whatever reason. The most obvious example would be when a connection to the database cannot be obtained (e.g. due to network or infrastructure issues). An improvement that has been made in JMS 2.0 is that the number of times a message has been processed can be retrieved.

For this we’re going to update the onMessage() received bean. The code below throws a dummy exception to return the message back to the queue. After the 3rd attempt, it writes the message contents to the server.log for later processing.

By updating the receiving code to look like this, we can see the message redelivery count increasing:

String msg;
try {
   msg = ((TextMessage) message).getText();
   int deliveryCount = message.getIntProperty("JMSXDeliveryCount");
   String deliveryCountStr = "Received this message " + deliveryCount + " times.";
   Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, deliveryCountStr);

   if (deliveryCount == 2) {
      Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, "Message can''t be processed. Details...: {0}", msg);
   } else {
      Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, "Throwing a dummy runtime exception");
      throw new RuntimeException("Dummy Exception");
} catch (JMSException ex) {
   Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, null, ex);

And the truncated version of the server.log file now looks like this (removing the stack traces for brevity):

SEVERE:   Received this message 1 times.
SEVERE:   Throwing a dummy runtime exception
WARNING:   MQJMSRA_MR2001: run:Caught Exception from onMessage():Redelivering:
SEVERE:   Received this message 2 times.
SEVERE:   Message can’t be processed. Details…: My JMS 2.0 Message

A more suitable use case for myself in this scenario is actually getting the message queue to stop processing messages in this scenario. I’d like then to build up whilst the reason for failure is detected (normally due to it being a networking / database issue).

I’ve yet to find something that I’m completely comfortable with, but provided this message doesn’t return, no further messages will be process. So prior to throwing a RuntimeException, I can put in place a Thread.sleep(milliseconds) to sleep for a period of time.

Logger.getLogger(NewMessageBean.class.getName()).log(Level.SEVERE, "Throwing a dummy runtime exception");
throw new RuntimeException("Dummy Exception");

This would allow messages to build up. The time to hold the queue for would need to be specific for the scenario being catered for. Ultimately it could be for an hour or so, and then remove the check for the number of times the message has been processed and just allow it to continue re-processing the message.

3. Monitoring the Queue(s)

3.1 imqadmin Tool

I said earlier that there is no default out-of-the-box monitoring within GlassFish for JMS. Thats not strictly true. Unfortunately (unless I’m missing something?) its not available in the Admin Console.

GlassFish does come bundled with a number of command line utilities, one of which being the Admin Manager for MQ. This utility can be found in \glassfish4\mq\bin and launched via imqadmin.


You then need to add a broker to localhost (Right-click on Brokers and Add…). The settings should be localhost, 7676, admin and admin (unless you’ve changed your localhost Admin password). Once added, you can then click the ‘Physical Destinations’ under your broker and the testQueue created earlier will appear.

By selecting the queue, and then clicking Properties (either via the Icon on the Toolbar or via Actions-Properties menu). This will show you details about the queue, including the number of messages sitting in it (0 at this stage).

To show some messages sitting in the queue, we can head back to our code and rename the queue being processed by the Message Bean to be testQueue2. Then running the sendMessage servlet we can populate the queue, with no listeners to pick the messages up. (For this to deploy correctly, you’ll need to create testQueue2 as per steps at the start of this post) (Further note – I also had to restart GlassFish after creating the 2nd queue – the broker and the application got completely confused and started only displaying the new queue?!? – Yet another reason to re-start GF?)

@MessageDriven(activationConfig = {
@ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "destinationLookup", propertyValue = "jms/testQueue2")

By running the send message servlet 5 times and then restarting the imqadmin client and connecting to localhost and viewing properties for queueTest I can see 5 message waiting in the queue.


3.2 Monitoring via a Servlet

This all seems quite long winded, so I put together a little servlet that would do something similar, using the QueueBrowser class. The QueueBrowser allows you to peek at the Message Queue without stripping the messages out.

I’ve yet to discover a way of listening all the queues on the server, but can’t imagine this would be difficult. Could quite easily create a JSP to ask the user to enter the queue name and then retrieve the messages and count from this servlet with some simple modifications.

try {
   QueueBrowser queueBrowser = jmsContext.createBrowser(queue);
   out.println("Queue Name: " + queueBrowser.getQueue().getQueueName() + "<br/>");
   Enumeration messages = queueBrowser.getEnumeration();
   int count = 0;
   while (messages.hasMoreElements()) {
      TextMessage msg = (TextMessage) messages.nextElement();
      out.println("Message [" + count + "] contains " + msg.getText() + "<br/>");
   out.println("Number of messages = " + count);
} catch (JMSException ex) {
   Logger.getLogger(queueViewer.class.getName()).log(Level.SEVERE, null, ex);
} finally {

4. Safety of Messages

One benefit from my experience in the MS world of MSMQ is the way messages are safely stored in cash of unexpected system restarts etc. With my 5 messages safely in the queue I’m going to restart GlassFish to see what happens to them.

And I’m pleased to say its kept them safe.


5. Conclusion

All in all this has been a whirlwind tour of JMS in general, but with a glance towards some of the new functionality in JMS 2.0. If anything is wrong in this post please let me know – I’m completely acknowledging my ‘learning’ position with regards to this technology.

Likewise, if anything is missing that you’d like me to research and cover, please let me know.

I’m already looking forward to the next GlassFish Users Group next month.

Evernote–My Tips and Set-up

I love Evernote. So much so some of my colleagues are wondering if I’m working on the side as a salesperson for them. Anyway, I saw a tweet in my timeline this morning asking for some tips for usage, so I thought I’d share my set-up and also some tips I gleaned along the way. I did a short post back in March about this topic, but I’ve moved on slightly since then, so this is a much fuller description and its been in place for about 4 months now.

Getting Things into Evernote

1. Your Evernote account has an email address. Its a a strange mix of characters, presumably representing an underlying unique username in their system. Too difficult to remember at any stage. So add it to your email contacts list and give it a friendlier name. Having my own domain, I’ve even set-up just so others can email me direct to Evernote when in meetings.

To get your Evernote email go to Account Settings on the Web, or in the Windows Desktop App – Tools –> Account Settings.

There is also a plug-in for Outlook so you can just click a button to add any email to Evernote.


Its been a while since I installed it, but a quick Google takes me to this discussion page, which should provide further help.

There are also useful annotations that you can add to the subject line to get the email into the right notebook in Evernote. By default it goes into your Inbox/default notebook. You can alter this by (using a @ to change the notebook) or add a tag (adding a #). You can also add reminders this way with an !. I very rarely use these though. For fuller, more precise details see here in this video.

2. The Android Widget. (Probably available on iOS too?). Normally when I’m walking around is when I remember things I need to do. This little widget will allow me to take notes either written, via the camera or an audio recording and send it straight to Evernote. It also records the GPS of where you were – should that be a factor.


(Also worth noting from here, you can add a PIN to your Evernote Mobile App to stop access in case it falls into the wrong hands. Short video here. Is this a premium only feature?)

3. Camscanner (Android App – iOS?). This is basically a scanner for your phone. It allows you to adjust what you’ve captured and then share via a range of ways, including to Evernote. As another aside, I’ve found this incredibly useful when attending conferences / lectures and taking photos of the presentation slides.

Setting Up Evernote – Notebooks and Stacks

A caveat – this is my set-up. It works for me. Its not necessarily the way. Or even the way for you.

1. Current Notebooks

My framework is based loosely on the GTD principles from David Allen. At the top of my notebook view I have 5 main notebooks – !Inbox, !IncubationIdeas, !Journal, !ToDo, !WaitingFor.


You’ll notice the ! marks in front. This is on purpose. It means these will stay above all my other notebooks. These are probably my most used notebooks.

  • Inbox represents things that I’ve captured but not yet processed.
  • Incubation Ideas are ideas for things I’d like to do. At some point. I may not have time now, or the time might not be quite right.
  • Journal is my attempt to capture everything I do in a day. (More on this later).
  • To Do is my main go to area. I have a separate note in here for each of the tasks that I’m aware of need action. (The title of the note is usually the next possible action and the note itself contains a checkbox (more later)).
  • Waiting For is stuff I’ve delegated but need to have an awareness of.

2. Current Project Notebooks

Sticking with the GTD principles I have a list of current projects that are happening in my life – ranging from home personal stuff to work related. Within these I tend to have at least 2 notes – ‘Project Log’ and ‘Project Actions’. Alongside these 2 notes sit anything else I’d like to refer to for future knowledge.


  • Project Log is something I’m experimenting with. Its essentially like a blog where I put date ordered notes about anything that happened, who was involved. It gives me a nice overview of everything that has happened in a project. (I used to keep these as separate notes, but found it a bit unwieldy).
  • Project Actions are a collect of To Do items that relate to this project. Again each action is annotated with an empty checkbox. It doesn’t always play nicely with my !ToDo notebook above, but I use the Search facility to overcome this generally (more later)

3. Other Notebook Stacks

Everything else is just for reference – stuff I may want to refer to in future or a way of storing things. This essentially takes a format of a file system – using notebooks as a folder equivalent.

Projects that are complete are moved into this area too for archiving / future referencing.

This for me is where Evernote comes into its own. The search facility is truly amazing, and was perhaps the part of Evernote that I really didn’t fully appreciate when first starting out.

Other Little Tips I’ve Gathered

1. Daily Journal. Each day I start a new note. Its created via a IFTTT recipe I’ve got in place so its automatically created and ready to use. The basic format is as below:


Before really starting work each day I note down the 3 overriding actions I’d like to get completed today – this comes from the principles outlined in Getting Results the Agile Way. I also note down any meetings, bullet point any actions I’ve performed during the day and I attempt to keep a food diary.

2. Search for Empty Checkboxes. Anything that requires action by me (either in my ToDo notebook or under one of my project notebooks) has a checkbox put against it. Once its complete I check the checkbox. This allows me to search for outstanding actions. To create this search (in the latest Windows Desktop App) click File->New Saved Search. Give it a title and set the criteria to:


3. Tags. I have used. And then I’ve removed. And then I’ve added back. Currently I’m not using them. I can see their benefit but it just adds an extra layer of complexity to my set-up.

4. Subscribe to your blog via Evernote Email. Have your own blog? Then subscribe to it using your Evernote email. That way every time a blog post is made, you’ll have an email back-up copy in your Evernote without having to do anything.

Performance Testing with JMeter 2.9 Book Review

Disclaimer: I was provided with a copy of the book free of charge in exchange for a review. The views expressed below are mine.


I’ve blogged previously about my love for the simplicity of JMeter, but also acknowledged my lack of in-depth know-how and its something I should really get more to grips with. I hoped that this book would take me to the next level with JMeter, and indeed the book arrived at a very timely stage for a project I’m currently working on.

The book begins with a very good introduction into why you’d want to use JMeter (or similar tools) in the first place, and presents a scenario where the benefits of it use can be seen. It then proceeds to guide through the installation of JMeter. All good so far. Unfortunately it then starts to go into what I refer to some more advanced topics before starting to use the application for the first time. These topics deserve a mention, but perhaps appear at the wrong place in the book. However glossing over them and making a mental note for where they exist for future reference seems to suffix.

The book however did present a few nuggets of information for myself, broadly split into the following categories:

  • How to record a test plan to simulate proper use of a website
  • The session / authorisation components
  • The distributed testing model to overcome local machine performance issues
  • The ability to directly test a database connection

And for these reasons alone has proved a very worthwhile read for myself. The book also goes into detail on a number of other topics / areas, many unfortunately not relevant to me at this moment, but were interesting none the less, and just re-confirmed my initial impressions that JMeter is very powerful tool with flexibility to accommodate many situations. I’ve made a mental note of the topics covered and will no doubt refer to this book to exploit them should the need arise on future projects.

Whilst the book is very good overall, there are a couple of weaknesses that need to be highlighted. I’ve bought a number of titles in the past from PacktPub, but they’ve lacked the final polish that other publishers appear to have. This also appears apparent in this book – indeed the first step-by-step exercise that appears could be set out so much clearer with just a little thought into the layout and numbering of the steps. At one point in another exercise I found myself referring to videos on YouTube for clarity in order to overcome the problem I was facing. However, whilst this could be classed as a weakness, some (including myself) may see this as an advantage – if everything was plain sailing would we really be engaging our minds and learning?

So all-in-all, I thought a very good book. I’ve certainly learnt a lot about a very useful tool, and its encouraged me to make more use of it on current and future projects. I now feel in a position to exploit its usefulness ever further than the simple tests I had been running previously. Indeed this week I think I’ve used JMeter more times and to better effect then I have since my original JMeter blog post back in 2010.