Saturday, March 28, 2015

How to configure IBM MQ 8 With WSO2 ESB

In this blog post, we will look at how to configure IBM MQ version 8 with WSO2 ESB and implement a proxy service to consume messages from a queue in IBM MQ.

Following are the steps we need to follow in order to configure ESB and implement our proxy service. 


1. Create the relevant JMS Administrative objects in IBM MQ.
2. Generate the JNDI binding file from IBM MQ
3. Configure WSO2 ESB JMS transport with the generated binding file and connection factory information.
4. Implement the proxy service and deploy it.
5. Publish a message to MQ and observe how it is consumed by ESB.

Create Queue Manager and Queue and Server Connection Channel in MQ

Step1.

Start the Web Sphere MQ Explorer. If you are not running on an administrator account, right click on the icon and select Run as Administrator option.


Step 2.

Click on the Queue Managers and Select New => Queue Manager to create a new queue manager.

We will name the queue manager as ESBQManager. Select create server connection channel option as you pass through the wizard with next button. You will get the option to specify the port this queue manager will use. Since we do not have any queue managers at the moment, we can use the default 1414 port.







Now we have created a queue manager object. Next we need to create a local queue which we will used to publish massages and consume from ESB. Lets name this queue as LocalQueue1.

Expand newly created ESBQManager and click on Queues and select New => Local Queue.





We will use default options for our local queue.

Next we need to create a server connection channel which will be used to connect to the queue manager.

Select Channels => New => Server-connection Channel option and give the channel name mychannel. Select default options for creating the channel.




Now we have created our queue manager, queue and server connection channel.

Generating the binding file

   Next we need to generate the binding file which will be used by IBM MQ client libraries for JNDI Look-up.  For that, we need to first create a directory where this binding file will be stored. I have created a directory named G:\jndidirectory for this purpose. 

Now go to MQ Explorer, click on JMS Administered Objects and select Add Initial Context.



In the connection details wizard, select File System option and browse to our newly created directory and click next and click finish.


Now, under the JMS Administered objects, we should be able to see our file initial context.



Expand it and click on Connection Factories to create a new connection factory.



We will name our connection factory as MyQueueConnectionFactory. For the connection factory type, select Queue Connection Factory.




Click next and click finish. Now Click on the newly created Connection Factory and select properties. Click on the connections option, browse and select our queue manager. You can also configure the port and the host name for connection factory. Since we used default values, we do not need to do any changes here. 






For the other options, go with the defaults. Next , we need to create a JMS Destination. We will use the same queue name LocalQueue1 as our destination and map it to our queue LocalQueue1 . Click on Destinations and select New => Destination. and provide name LocalQueue1. When you get the option to select the queue manager and queue browse and select ESBQManager and LocalQueue1 .





Now we are done with creating the Initial Context. If you now browse to the directory we specified, you should be able to see the newly generated binding file.



In order to connect to the Queue, we need to configure channel authentication. For the ease of use, lets disable channel authentication for our scenario. For that run the command runmqsc from the command line and execute the following two commands. Note that you have to start command prompt as admin user.

runmqsc ESBQManager

ALTER QMGR CHLAUTH(DISABLED)

REFRESH SECURITY TYPE(CONNAUTH)


Now we are done with configuring the IBM MQ.  

Configuring WSO2 ESB JMS Transport. 


open axis2.xml found in wso2esb-4.8.1\repository\conf\axis2 directory and add the following entries to it near the commented out jms transport receiver section.

<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
  <parameter name="default" locked="false">
    <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
    <parameter name="java.naming.provider.url" locked="false">file:/G:/jndidirectory</parameter>
    <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">MyQueueConnectionFactory</parameter>
    <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
    <parameter name="transport.jms.UserName" locked="false">nandika</parameter>
    <parameter name="transport.jms.Password" locked="false">password</parameter>
  </parameter>

  <parameter name="myQueueConnectionFactory1" locked="false">
    <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
    <parameter name="java.naming.provider.url" locked="false">file:/G:/jndidirectory</parameter>
    <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">MyQueueConnectionFactory</parameter>
    <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
    <parameter name="transport.jms.UserName" locked="false">nandika</parameter>
    <parameter name="transport.jms.Password" locked="false">password</parameter>
  </parameter>
</transportReceiver>

Similarly add jms transport sender section as follows.

<transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender">
  <parameter name="default" locked="false">
    <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
    <parameter name="java.naming.provider.url" locked="false">file:/G:/jndidirectory</parameter>
    <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">MyQueueConnectionFactory</parameter>
    <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
    <parameter name="transport.jms.UserName" locked="false">nandika</parameter>
    <parameter name="transport.jms.Password" locked="false">password</parameter>
  </parameter>

  <parameter name="myQueueConnectionFactory1" locked="false">
    <parameter name="java.naming.factory.initial" locked="false">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
    <parameter name="java.naming.provider.url" locked="false">file:/G:/jndidirectory</parameter>
    <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">MyQueueConnectionFactory</parameter>
    <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
    <parameter name="transport.jms.UserName" locked="false">nandika</parameter>
    <parameter name="transport.jms.Password" locked="false">password</parameter>
  </parameter>
</transportSender>

Since we are using IBM MQ queue manager default configuration, it is expecting username password client authentication. Here, the username and password is the login information of your logged in operating system account.


Copy MQ client libraries to respective directories.


Copy jta.jar and jms.jar to repository/components/lib directory.
Copy com.ibm.mq_2.0.0.jar and fscontext_1.0.0.jar to repository/components/dropins directory. Download the jar files from here.

Deploy JMSListener Proxy Service.

Now start esb and deploy the following simple proxy service. This proxy service act as a listener to our queue LocalQueue1 and when ever we put a message to this queue, the proxy service will pull that message out of the queue and log it.

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="MyJMSProxy"
       transports="jms"
       startOnLoad="true"
       trace="disable">
   <description/>
   <target>
      <inSequence>
         <log level="full"/>
         <drop/>
      </inSequence>
   </target>
   <parameter name="transport.jms.Destination">LocalQueue1</parameter>
</proxy>

Testing our proxy service

Go to MQ Explorer and add a message to local queue. 



Now you will be able to see the message logged in ESB console as well as in the log file.

Enjoy JMS with IBM MQ

22 comments:

  1. Nice post. Will be very useful..!!

    ReplyDelete
  2. Hello Nandika thanks for this post. Im trying to achieve this by deploying ESB on my Machine and MQ on a Win 2012 server on the same network segment nonetheless when I start ESB I'm receiving the following errors:

    JMSListener Unable to continue server startup as it seems the JMS Provider is not yet started. Please start the JMS provider now.
    ERROR JMSListener Connection attempt : 5 for JMS Provider failed. Next retry in 320 seconds

    What am I doing wrong ? Is this tutorial meant to be deployed both ESB and MQ on the same machine to make it work or Should I configure something else on the MQ side to make ESB establish remote connection ?

    Thanks in advance.

    ReplyDelete
  3. Hi Rafael,

    The JMSListener unable to connect error occurs when the security configurations are not correct. In my scenario, I disabled the channel authentication using ALTER QMGR CHLAUTH(DISABLED). However, in your case, you will need to configure authentication properly for the MQ Server.

    Following video is a good starting point.

    https://www.youtube.com/watch?v=81zxSdwm4ek

    ReplyDelete
    Replies
    1. Hello Nandika and thanks for your quick answer. I'll take a look at it asap. So basically I need to configure channel security so I can connect remotely to MQ... I'll comeback here if needed.

      Thx

      Delete
  4. Hey Nandika

    I am struggling to find the value for provider.url when the MQ server is installed on the remote box.

    thanks
    Samta

    ReplyDelete
  5. provider url is the local directory containing the .binding file you have generated

    Regards
    Nandika

    ReplyDelete
  6. Very nice post!

    How would you implement a proxy service, not to consume, but to send messages to WMQ ?

    ReplyDelete
    Replies
    1. I've achieved this by implementing this guide and the WMQ and WSO2ESB in the same machine, I want to do it remotely but I guess I need to learn WMQ deeply. If I find the config files I'll post then here so you can see, but It's pretty straighfordward.

      Delete
  7. I'm also getting the same exception:
    2015-08-04 14:57:01,549 [-] [pool-12-thread-1] INFO JMSListener JMS listener started
    2015-08-04 14:57:01,550 [-] [pool-12-thread-1] DEBUG JMSEndpoint JMS destination type not given. default queue
    2015-08-04 14:57:01,551 [-] [pool-12-thread-1] DEBUG JMSEndpoint JMS reply destination type not given. default queue
    2015-08-04 14:57:01,565 [-] [pool-12-thread-1] DEBUG JMSUtils Creating a GenericConnection using credentials : (svctestgenmq/xxxxxx)
    2015-08-04 14:57:03,005 [-] [pool-12-thread-1] ERROR JMSListener Unable to continue server startup as it seems the JMS Provider is not yet started. Please start the JMS provider now.
    2015-08-04 14:57:03,006 [-] [pool-12-thread-1] ERROR JMSListener Connection attempt : 1 for JMS Provider failed. Next retry in 20 seconds

    My WSO2 and WMQ installations are on different machines. I disabled the channel authentication, but still getting the same error. Any idea about what could be the problem ?

    ReplyDelete
  8. Aug 11, 2015 2:53:38 AM MQAdapter setMQListener
    INFO: QS9ESG00:SG/QS9HV.GEBFORBR.TRADEFINANCE.IN:--------Start MQ setMQListener-
    ---------------
    Aug 11, 2015 2:53:39 AM MQAdapter setMQListener
    SEVERE: JMSWMQ2013: The security authentication was not valid that was supplied
    for QueueManager 'QS9ESG00' with connection mode 'Client' and host name 'XISXXXXX(11416)'.
    com.ibm.msg.client.jms.DetailedJMSSecurityException: JMSWMQ2013: The security authentication was not valid that was supplied for QueueManager 'QS9ESG00' with connection mode 'Client' and host name 'XISXXXXX(11416)'.
    Please check if the supplied username and password are correct on the QueueManager to which you are connecting.
    at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:514)
    at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:214)
    at com.ibm.msg.client.wmq.internal.WMQConnection.(WMQConnection.java:406)
    at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:6865)
    at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:6221)
    at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl._createConnection(JmsConnectionFactoryImpl.java:285)
    at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:233)
    at com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6016)
    at com.ibm.mq.jms.MQQueueConnectionFactory.createQueueConnection(MQQueueConnectionFactory.java:111)
    at MQAdapter.reqConnection(MQAdapter.java:84)
    at MQAdapter.setMQListener(MQAdapter.java:111)
    at MQAdapter.main(MQAdapter.java:550)
    Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2035' ('MQRC_NOT_AUTHORIZED').
    at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:202)
    ... 10 more
    I have ALTER CHLAUTH(DISABLED) and refreshed the security, but still getting above exception. Believed, this issue is related with MQAdmin authentication.. ?
    Perhaps, I am using old jta.jar, connector.jar & dhcore.jar for my standalone program to connect with MQ manager. pls suggest me, asap.

    ReplyDelete
  9. I get with MQ8:

    Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '6114' ('MQRC_INSUFFICIENT_DATA').
    at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:209)
    ... 29 more
    Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=6114
    at com.ibm.mq.ese.intercept.JmqiOpenInterceptorImpl.processExceptionAfterOpen(JmqiOpenInterceptorImpl.java:696)
    at com.ibm.mq.ese.intercept.JmqiOpenInterceptorImpl.afterSpiOpen(JmqiOpenInterceptorImpl.java:212)
    at com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.spiOpen(InterceptedJmqiImpl.java:452)
    at com.ibm.mq.ese.jmqi.ESEJMQI.spiOpen(ESEJMQI.java:513)
    at com.ibm.msg.client.wmq.internal.WMQMessageProducer$SpiIdentifiedProducerShadow.initialise(WMQMessageProducer.java:762)

    I think that means the drivers are incompatible with MQ8. Any ideas? I also tried to use the jars that come with MQ8 (and also work with simple JMS client) but I get an error about missing MQQueueConnectionFactoryFactory.

    ReplyDelete
    Replies
    1. I also wrote a simple JMS client that uses the jar attached. I get a class not found exception on com.sun.jndi.fscontext.RefFSContextFactory. If I use the drivers that come with MQ8 my simple JMS client works fine. If I use those same MQ8 drivers with WSO2 I also get the above error. seems like dropins is stuck. I'm using puppet to build a wso2 install. I started the build from scratch and still get the above errors with MQ8 drivers and the ones attached to this article.

      Delete
    2. Jeremiah did you ever figure this out? I am having the same issues @Jeremiah Johnson

      Delete
    3. You need to use the jars from the git repo provided by Nandika

      Delete
  10. This comment has been removed by the author.

    ReplyDelete
  11. HI Nandika ,

    In my environment JMS is not created to connect to ESB to MQ Cluster.

    And how can we create using MQ Client and ESB to connect IBM MQ Cluster

    ReplyDelete
  12. Nandika, Can you share your axis2.xml file related to this section?

    ReplyDelete
  13. Hi Nandika,
    im getting an error at this step

    REFRESH SECURITY TYPE(CONNAUTH)
    2 : REFRESH SECURITY TYPE(CONNAUTH)
    AMQ8405: Syntax error detected at or near end of command segment below:-
    REFRESH SECURITY TYPE(CONN

    AMQ8427: Valid syntax for the MQSC command:

    REFRESH SECURITY [ (*) ]
    TYPE ( AUTHSERV | SSL )

    I'm using IBM MQ ver 7.5

    can u guid me with this?

    ReplyDelete
    Replies
    1. Hi Anushanth,
      This error seems to be not bothering anyway. I just neglected above error and restarted MQ application and did the next steps. Everything worked as expected.

      Delete
  14. Hi Nandika,
    Thanks a lot for this post. I am new to WSO2 and this post helped me a lot in setting up the initial JMS connection with WMQ 8. I was able to do the one way flow using this. However when I am trying to create a flow where I want to make a request to backend with WMQ 8, its not working though I can see the response in the reply to queue my flow is not picking it. In the logs I can see the below errors.
    I was wondering if you can help me in this.

    Regards,
    Gaurav


    ERROR {org.apache.synapse.core.axis2.AsyncCallback} - com.ibm.mq.jms.MQSession cannot be cast to javax.jms.QueueSession {org.apache.synapse.core.axis2.AsyncCallback}
    java.lang.ClassCastException: com.ibm.mq.jms.MQSession cannot be cast to javax.jms.QueueSession
    at org.apache.axis2.transport.jms.JMSUtils.createConsumer(JMSUtils.java:531)
    at org.apache.axis2.transport.jms.JMSSender.waitForResponseAndProcess(JMSSender.java:306)
    at org.apache.axis2.transport.jms.JMSSender.sendOverJMS(JMSSender.java:283)
    at org.apache.axis2.transport.jms.JMSSender.sendMessage(JMSSender.java:169)
    at org.apache.axis2.transport.base.AbstractTransportSender.invoke(AbstractTransportSender.java:112)
    at org.apache.axis2.engine.AxisEngine$TransportNonBlockingInvocationWorker.run(AxisEngine.java:626)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

    ReplyDelete
  15. You might also need to disable connection auth

    ALTER QMGR CHLAUTH(DISABLED) CONNAUTH(' ')
    REFRESH SECURITY TYPE(CONNAUTH)

    ReplyDelete