Project

General

Profile

Image Support #3689

JBOSS EAP 7.2.9 : Unable to listen message from tibco ems queues

Added by Shweta about 2 months ago. Updated 4 days ago.

Status:
Feedback
Priority:
Normal
Assignee:
Due date:
% Done:

0%

Operating System:
Linux
JRE:
Not Applicable
Instance Type:
Not Applicable
Your Marketplace Account ID:
2872-5568-4589
Marketplace:
Amazon Web Services
Customer State:
Sydney
Customer Country:
New South Wales

Description

Hi Team,

We are integrating jboss with tibco ems queues. Please look at the below issues and assist us.

1. With the current configuration , our ec2 server is able to establish the live connection with EMS-BB server. They pushed the test message from their end to our server but we are not able to listen that message. Our request queues are not able to connect and message seems to be pending at their end. Please find the attached standalone-full file and let us know if any configuration is missed and in which subsytem queue entries needs to be configured.

2. Also, as discussed in the following ticket Image Support #3666 : JBOSS EAP 7.2.0 : TIBCO queues configuration using existing jms .rar (https://support.midvision.com/issues/3666) , we are getting below error :
2020-11-28 02:08:04,499 DEBUG [org.jboss.jca.core.recovery.DefaultRecoveryPlugin] (Periodic Recovery) Error during connection close(): java.lang.reflect.InvocationTargetException Caused by: java.lang.NullPointerException
at //org.jboss.resource.adapter.jms.JmsSession.close(JmsSession.java:297)
... 13 more

for which we have received an update that "JNDI name defined" .Could you please refer configuration file and error logs attached and let us know in which subsytem JNDI needs to be defined.

3. EMSBB team have noticed that multiple jms sessions are getting created after 5 minute interval of time. Can you please check the attached tibco server logs1.txt file and help us understand such behavior.

Please let us know at the earliest as we are not able to proceed and this is a blocker for us.

Also , Could you please get us someone from Red Hat Support Team to have session over call to discuss further.

Thanks,
Shweta Hegade,IPAM2


Files

standalone-full.xml_tibcoQueues (45.5 KB) standalone-full.xml_tibcoQueues Standalone-full configuration file Shweta, 11/30/2020 03:55 PM
tibco server logs.txt (26.1 KB) tibco server logs.txt server.log Shweta, 11/30/2020 03:56 PM
tibco server logs 1.txt (38.4 KB) tibco server logs 1.txt Multiple jms sessions in server.log Shweta, 11/30/2020 04:04 PM
tibco server logs 3.txt (6.75 KB) tibco server logs 3.txt Shweta, 12/01/2020 03:16 PM
ClassCastException.txt (7.74 KB) ClassCastException.txt Shweta, 12/01/2020 04:43 PM
standalone-full.xml (43.8 KB) standalone-full.xml Mariusz, 12/01/2020 04:51 PM
jboss-generic-jms-ra.zip (44 KB) jboss-generic-jms-ra.zip Mariusz, 12/01/2020 04:51 PM
tibco javax.naming.ServiceUnavailableException.txt (9.76 KB) tibco javax.naming.ServiceUnavailableException.txt Shweta, 12/02/2020 11:39 AM
tibco ssl error 2020-12-09.txt (1.29 MB) tibco ssl error 2020-12-09.txt Shweta, 12/09/2020 10:19 AM
tibco ssl error 2020-12-11.txt (738 KB) tibco ssl error 2020-12-11.txt Shweta, 12/10/2020 03:04 PM
jboss system properties.txt (3.43 KB) jboss system properties.txt Shweta, 12/11/2020 11:25 AM
tibco server log 5.txt (11.9 KB) tibco server log 5.txt Shweta, 12/16/2020 03:11 PM
standalone-full.xml (43.4 KB) standalone-full.xml Shweta, 12/29/2020 07:43 AM
#1

Updated by Mariusz about 2 months ago

  • Status changed from New to In Progress
  • Assignee changed from Mariusz to Red Hat Support
  • Priority changed from Blocker to High

Hi Schweta,

If I understand correctly this ticket is actually a duplication of previously created ticket #3666.
I sent your update to the Red Hat Support team and attached the files you shared with us.

As I mentioned in my direct email when you run an instance based on one of our images you are automatically entitled to "standard" support.

See more about our support options:
https://www.midvisioncloud.com/support-services/support-slas/

As you will see there is no telephone support included.
Also, the highest support ticket priority with standard support is ‘High’, so I am changing the priority to High.

I would like to ensure you, that from our side we do what we can to help you resolve this issue. Your comments requests and attachments are immediately shared with the Red Hat Support team.

Regards,
Mariusz Chwalek

#2

Updated by Mariusz about 2 months ago

  • Parent task set to #3666
#3

Updated by Mariusz about 2 months ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Reply from Red Hat Support team:

The errors you see have nothing to do with you application. They come from XA recovery probably to to the fact that you have multiple JNDI servers in your JNDI URL.
Please not that JBOSS Does not connect to TIBCO that is don in your application code. So if you have any issues with message consumption/production we would have to look at your application.

Can you attach a sample of you code that is having problems?

Your configuration seems fine.

Best regards - Tom Ross

#4

Updated by Shweta about 2 months ago

Hi Tom,

As suggested we have configured below @ActivationConfigProperty & @ResourceAdapter in our application code in MDB class and deployed the ear on jboss :

@MessageDriven(name = "SearchSubnetMDB",activationConfig = {
@ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "destination", propertyValue = "java:global/tibco/jms/queue/Telstra.IPAM.SearchSubnet.Request"),
@ActivationConfigProperty(propertyName = "connectionFactory", propertyValue = "java:/jms/XAQueueConnectionFactory"),
@ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
@ActivationConfigProperty(propertyName = "useJNDI", propertyValue = "true")})
@ResourceAdapter("genericjms-xa.rar")

The same destination and connectionFactory has been configured in the standalone-full.xml. But we are getting below error :

02:01:55,536 ERROR [org.jboss.resource.adapter.jms.inflow.JmsActivation] (default-threads - 1) Unable to reconnect org.jboss.resource.adapter.jms.inflow.JmsActivationSpec@607243f4(ra=org.jboss.resource.adapter.jms.JmsResourceAdapter@4ddf3831 destination=java:global/tibco/jms/queue/Telstra.IPAM.SearchSubnet.Request destinationType=javax.jms.Queue acknowledgeMode=Auto-acknowledge subscriptionDurability=false reconnectInterval=10 reconnectAttempts=-1 user=null maxMessages=1 minSession=1 maxSession=15 connectionFactory=java:/jms/XAQueueConnectionFactory): javax.naming.NameNotFoundException: Name not found: 'jms/queue/Telstra.IPAM.SearchSubnet.Request'
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:713)
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:489)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at

The detailed log is attached. Could you please check and let us know how to resolve this error.

Could you please let us how to configure myQueue entries in standalone-full.xml.

Thanks,
Shweta Hegade,IPAM2

#5

Updated by Mariusz about 2 months ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#6

Updated by Shweta about 2 months ago

Hi Mariusz,

With the below configuration in MDB class at application level code javax.naming.NameNotFoundException: Name not found: 'jms/queue/Telstra.IPAM.SearchSubnet.Request' error has been resolved but now we are getting different error.

error :
03:41:25,932 ERROR [org.jboss.resource.adapter.jms.inflow.JmsActivation] (default-threads - 1) Unable to reconnect org.jboss.resource.adapter.jms.inflow.JmsActivationSpec@1bd0029(ra=org.jboss.resource.adapter.jms.JmsResourceAdapter@bc7d096a destination=java:global/tibco/Telstra.IPAM.SearchSubnet.Request destinationType=javax.jms.Queue acknowledgeMode=Auto-acknowledge subscriptionDurability=false reconnectInterval=10 reconnectAttempts=-1 user=null maxMessages=1 minSession=1 maxSession=15 connectionFactory=java:global/tibco/XAQueueConnectionFactory): java.lang.ClassCastException: class org.jboss.resource.adapter.jms.JmsConnectionFactoryImpl cannot be cast to class javax.jms.XAConnectionFactory (org.jboss.resource.adapter.jms.JmsConnectionFactoryImpl is in unnamed module of loader '' @45577d0b; javax.jms.XAConnectionFactory is in unnamed module of loader '' @36fd14dd)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.setupConnection(JmsActivation.java:439)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.setupConnection(JmsActivation.java:416)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.setupActivation(JmsActivation.java:315)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.handleFailure(JmsActivation.java:258)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation$SetupActivation.run(JmsActivation.java:547)
at //org.jboss.jca.core.workmanager.WorkWrapper.runWork(WorkWrapper.java:445)
at //org.jboss.as.connector.services.workmanager.WildflyWorkWrapper.runWork(WildflyWorkWrapper.java:69)
at //org.jboss.jca.core.workmanager.WorkWrapper.run(WorkWrapper.java:223)
at //org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:29)
at //org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:789)
at //org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:44)
at //org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:809)
at java.base/java.lang.Thread.run(Thread.java:834)
at //org.jboss.threads.JBossThread.run(JBossThread.java:485)

@ActivationConfigProperty & @ResourceAdapter at code level :
@MessageDriven(name = "SearchSubnetMDB",activationConfig = {
@ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "destination", propertyValue = "java:global/tibco/Telstra.IPAM.SearchSubnet.Request"),
@ActivationConfigProperty(propertyName = "connectionFactory", propertyValue = "java:global/tibco/XAQueueConnectionFactory"),
@ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
@ActivationConfigProperty(propertyName = "useJNDI", propertyValue = "true")})
@ResourceAdapter("genericjms-xa.rar")

Please find the attached logs for your reference and let us know how to proceed with the error resolution.

Thanks,
Shweta Hegade,IPAM2

#7

Updated by Mariusz about 2 months ago

Reply from IBM to update #4 :

Does the queue Telstra.IPAM.SearchSubnet.Request exist in TIBCO JNDI?
What's its JNDI name?

For example my MDB connects to inQueue so in TIBCO admin console I can do

tcp://aza:7222> show queue inQueue
 Queue:                 inQueue
 Type:                  static
 Properties:            *prefetch=5,maxRedelivery=15,maxMsgs=100000
 JNDI Names:            "jms/queue/inQueue" 
 Bridges:               <none>
 Receivers:             0
 Pending Msgs:          0, (0 persistent)
 Delivered Msgs:        0
 Pending Msgs Size:     0.0 Kb, (0.0 Kb persistent)
So I can see that the queue has JNDI name jms/queue/inQueue (please note that it won't work if you start your TIBCO JNDI names with "/").
So in my MDB I have

@MessageDriven(name = "GenericRAInQueueMDB", activationConfig = {
        @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
        @ActivationConfigProperty(propertyName = "destination", propertyValue = "jms/queue/inQueue"),
        @ActivationConfigProperty(propertyName = "jndiParameters", propertyValue = "java.naming.security.principal=${tibco.user};java.naming.security.credentials=${tibco.password};java.naming.factory.initial=com.tibco.tibjms.naming.TibjmsInitialContextFactory;java.naming.provider.url=tcp://aza:7222"),
        @ActivationConfigProperty(propertyName = "connectionFactory", propertyValue = "${tibco.qcf}"),
        @ActivationConfigProperty(propertyName = "user", propertyValue = "${tibco.user}"),
        @ActivationConfigProperty(propertyName = "password", propertyValue = "${tibco.password}"),
        @ActivationConfigProperty(propertyName = "maxSession", propertyValue = "${in.mdb.maxsession}"),
        @ActivationConfigProperty(propertyName = "reconnectAttempts", propertyValue = "${in.mdb.reconnect.attempts}")

})
If you want to inject the queue or connection factory to your code then you can use external context like below

So you define external JNDI context as

<subsystem xmlns="urn:jboss:domain:naming:2.0">
            <bindings>
                <external-context name="java:global/tibco" module="org.jboss.genericjms.provider" class="javax.naming.InitialContext">
                    <environment>
                        <property name="java.naming.factory.initial" value="com.tibco.tibjms.naming.TibjmsInitialContextFactory"/>
                        <property name="java.naming.factory.url.pkgs" value="com.tibco.tibjms.naming"/>
                        <property name="java.naming.provider.url" value="tcp://${tibco.host}:${tibco.port}"/>
                        <property name="java.naming.security.principal" value="${tibco.user}"/>
                        <property name="java.naming.security.credentials" value="${tibco.password}"/>
                        <property name="org.jboss.as.naming.lookup.by.string" value="true"/>
                    </environment>
                </external-context>
                <lookup name="java:/jms/tibco/queue/sourceQueue" lookup="java:global/tibco/jms/queue/sourceQueue"/>
                <lookup name="java:/jms/tibco/queue/targetQueue" lookup="java:global/tibco/jms/queue/targetQueue"/>
                <lookup name="java:/jms/tibco/queue/inQueue" lookup="java:global/tibco/jms/queue/inQueue"/>
                <lookup name="java:/jms/tibco/queue/outQueue" lookup="java:global/tibco/jms/queue/outQueue"/>
                <lookup name="java:/jms/tibco/topic/testTopic" lookup="java:global/tibco/jms/topic/testTopic"/>
                <lookup name="java:/jms/tibco/topic/inTopic" lookup="java:global/tibco/jms/topic/inTopic"/>
                <lookup name="java:/jms/tibco/topic/outTopic" lookup="java:global/tibco/jms/topic/outTopic"/>
            </bindings>
            <remote-naming/>
        </subsystem>
So you can do things like

@Resource(lookup = "${tibco.out.queue.fqn}")
private Queue outQueue = null;
Where tibco.out.queue.fqn=java:/jms/tibco/queue/outQueue

BTW "useJNDI" does not work with TIBCO.

I'm attaching my project and standalone-full.xml to the case.

Best regards - Tom Ross

#8

Updated by Mariusz about 2 months ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#9

Updated by Mariusz about 2 months ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Reply from Red Hat Support team to #5:

Hi,

I've provide you with correct configuration as per my attached project. The configuration that you have is not correct so please take a look at my attache project and make modifications.
As far as the cast exception goes - are you including javax.jms.* package twice? 

Could you attach to the case your module and the project ?

Best regards - Tom Ross

#10

Updated by Mariusz about 2 months ago

Reply from Red Hat

I looked at the source code and I can see that org.jboss.resource.adapter.jms.JmsConnectionFactoryImpl does not implement javax.jms.XAConnectionFactory hence the exception. 
Looking at the configuration I can't see where org.JBoss.resource.adapter.jms.JmsConnectionFactoryImpl since we don't expose it.
Best regards - Tom Ross
#11

Updated by Mariusz about 2 months ago

Another update from Red Hat:

I've created a simple KCS document [1] which has a simple project that can be deployed in JBoss EAP and consume message from TIBCO EMS.

Best regards - Tom Ross

[1] https://access.redhat.com/solutions/5619661

#12

Updated by Shweta about 2 months ago

Hi Mariusz,Tom,

Thank you so much for your update and assistance. With the modified configurations in application code , we are able to connect to tibco queue . We are able to consume messages and send response back to tibco queues.

Currently we are testing MASSL connectivity with EMSBB team but we are failed to connect to their provider url which is having ssl protocol.

Error :
22:26:12,687 ERROR [org.jboss.resource.adapter.jms.inflow.JmsActivation] (default-threads - 1) Unable to reconnect org.jboss.resource.adapter.jms.inflow.JmsActivationSpec@52979ca5(ra=org.jboss.resource.adapter.jms.JmsResourceAdapter@6a991f00 destination=Telstra.IPAM.SearchSubnet.Request destinationType=javax.jms.Queue acknowledgeMode=Auto-acknowledge subscriptionDurability=false reconnectInterval=10 reconnectAttempts=60 user=n109978 password=<not shown> maxMessages=1 minSession=1 maxSession=15 connectionFactory=n109978_IPAM2_QCF jndiParameters={java.naming.security.principal=n109978, java.naming.factory.initial=com.tibco.tibjms.naming.TibjmsInitialContextFactory, java.naming.provider.url=ssl://cly-prim1-dev.emsbb.telstra.com:7664,ssl://cly-prim1-dev.emsbb.telstra.com:8888,ssl://cly-prim2-dev.emsbb.telstra.com:8889,ssl://cly-prim2-dev.emsbb.telstra.com:7664, java.naming.security.credentials=<not shown>}): javax.naming.ServiceUnavailableException: Failed to query JNDI: Failed to connect to any server at: tcp://cly-prim1-dev.emsbb.telstra.com:7664, tcp://cly-prim1-dev.emsbb.telstra.com:8888, tcp://cly-prim2-dev.emsbb.telstra.com:8889, tcp://cly-prim2-dev.emsbb.telstra.com:7664 [Root exception is javax.jms.JMSException: Failed to connect to any server at: tcp://cly-prim1-dev.emsbb.telstra.com:7664, tcp://cly-prim1-dev.emsbb.telstra.com:8888, tcp://cly-prim2-dev.emsbb.telstra.com:8889, tcp://cly-prim2-dev.emsbb.telstra.com:7664]
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:669)
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:489)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.lookup(JmsActivation.java:564)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.setupDestination(JmsActivation.java:387)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.setupActivation(JmsActivation.java:314)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation.handleFailure(JmsActivation.java:258)
at //org.jboss.resource.adapter.jms.inflow.JmsActivation$SetupActivation.run(JmsActivation.java:547)
at //org.jboss.jca.core.workmanager.WorkWrapper.runWork(WorkWrapper.java:445)
at //org.jboss.as.connector.services.workmanager.WildflyWorkWrapper.runWork(WildflyWorkWrapper.java:69)
at //org.jboss.jca.core.workmanager.WorkWrapper.run(WorkWrapper.java:223)
at //org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:29)
at //org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:789)
at //org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:44)
at //org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:809)
at java.base/java.lang.Thread.run(Thread.java:834)
at //org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: javax.jms.JMSException: Failed to connect to any server at: tcp://cly-prim1-dev.emsbb.telstra.com:7664, tcp://cly-prim1-dev.emsbb.telstra.com:8888, tcp://cly-prim2-dev.emsbb.telstra.com:8889, tcp://cly-prim2-dev.emsbb.telstra.com:7664
at //com.tibco.tibjms.TibjmsConnection._create(TibjmsConnection.java:1393)
at //com.tibco.tibjms.TibjmsConnection.<init>(TibjmsConnection.java:4115)
at //com.tibco.tibjms.TibjmsQueueConnection.<init>(TibjmsQueueConnection.java:36)
at //com.tibco.tibjms.TibjmsxCFImpl._createImpl(TibjmsxCFImpl.java:200)
at //com.tibco.tibjms.TibjmsxCFImpl._createConnection(TibjmsxCFImpl.java:253)
at //com.tibco.tibjms.TibjmsQueueConnectionFactory.createQueueConnection(TibjmsQueueConnectionFactory.java:87)
at //com.tibco.tibjms.naming.TibjmsContext$Messenger.request(TibjmsContext.java:325)
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:655)
... 17 more

Could you please suggest if any other configuration is required at our end.

Thanks,
Shweta Hegade,IPAM2.

#13

Updated by Shweta about 2 months ago

Also, the emssbb team is fcaing following error at their end :
2020-12-02 22:38:50.179 SSL handshake failed: ret=-1, reason=unknown protocol
2020-12-02 22:38:50.179 [OpenSSL Error]: file=ossl.c, line=1767
2020-12-02 22:38:50.179 2:error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol:s23_srvr.c:643:

Please let us know if you could assist us with this error resolution.

Thanks,
Shweta Hegade,IPAM2

#14

Updated by Mariusz about 2 months ago

Reply from Red Hat:

Hi David,
I'm glad to hear that youare able to use TIBCO and JBoss EAP successfully.
I'm afraid I'm not in position to assist you with TIBCO SSL set up. I don't know what is required by TIBCO to add SSL configuration.
Generic JMS RA is just a wrapper around TIBCO java runtime so I'm not even sure if that would work with SSL.
If you have TIBCO documentation describing TIBCO java client SSL set up please point me to it and we would try it.

Best regards - Tom Ross

#15

Updated by Mariusz about 2 months ago

  • Parent task deleted (#3666)
#16

Updated by Shweta about 1 month ago

Hi Tom,

It would he helpful if you elaborate this statement "Generic JMS RA is just a wrapper around TIBCO java runtime so I'm not even sure if that would work with SSL." so if we are to proceed with generic jms ra then we wont be able to establish ssl connection with tibco queues ?

Also , please find the below configuration we received from ems team, Let me know if this helps :

ClientID =
Load Balanced = no
ssl_trusted = /data01/shared/EMS1/DE001-EPIC-EMS-SERVER-6/ems/certs/August2020/TelstraTestSHA2CAChain.cer.pem
ssl_expected_hostname = dev.cly5.emsbb.telstra.com
ssl_auth_only = enabled
tcp://lxapp4270.dc.corp.telstra.com:7464>

Let us know if you require any other information from our or ems end.

Thanks,
Shweta Hegade

#17

Updated by Shweta about 1 month ago

Please find below summary from ems side , please let us know if this helps to understand the configuration:
Tibco ems end configuration:
------- Summary ---------
SSL Protocol: TLSv1.2
Ciphers = /AES128-SHA:AES256-SHA256:AES256-SHA:AES256-GCM-SHA384
Certs SHA2 from PKI
Client sider cert verification is the chain of Root and Intermediate and not identity
server side cert verification is also the chain or Root and intermediate and not identity (all clients follow this)

Thanks,
Shweta Hegade

#18

Updated by Mariusz about 1 month ago

Hi Shweta,

TIBCO does not provide JCA resource adapter with their EMS system so if you want to connect to it from a java EE container like JBoss EAP you need a JCA resource adapter. That's where Generic RA comes in. It's a wrapper around TIBCO JMS run time that enables to expose TIBCO JMS connections as managed connections that JBoss can utilise. Red Hat does not test JBoss EAP and TIBCO over SSL so we don't if that combination would work since we only test JBoss EAP with TIBCO over non SSL connections. I said "I'm not even sure if that would work with SSL" because when I asked the Generic RA developer if that was possible he said that he wasn't sure and it all depended on how TIBCO handles SSL.
I looked around our QA infrastructure to see if we had anywhere TIBCO EMS instance with SSL but we don't, we only have non SSL.
The information that you provided from your EMS guys is insufficient. We would also need inforation about setting up TIBCO java run time with SSL.
If you were to provide information about
- how to set up TIBCO EMS broker with SSL (step by step)
- how to set up TIBCO EMS Java client with SSL (step by step)

then I could try to see if we can get it to work.

Hope this helps - Tom Ross

P.S.
Please note I'm off from today afternoon, back on 17 of December.

Tom

#19

Updated by Shweta about 1 month ago

Hi Tom ,
Thanks for the information provided. We have asked TIBCO EMS team to share requested details with us. We will provide the information once we have it.

We have below queries :
1. Does some extra configurations required in eap for establishing SSL connection with TIBCO ems as we can not proceed with non ssl connection if yes then could you please let us know.
2. Would there be any changes required in generic jms rar or is it fine to use the same one for ssl connection or any other customized Resource Adapter is required.

Please let us know.

@Mariusz , would you please request Red Hat Support team to provide any other resource as Tom Ross will not be available.

Thanks,
Shweta Hegade

#20

Updated by Mariusz about 1 month ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#21

Updated by Mariusz about 1 month ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Hi Shweta,

To answer your questions - most like there will be changes to the Generic RA configuration but at this stage we don't know what those change might be.
When it comes to providing support, someone will try to assist you while I'm away.

Best regards - Tom Ross

#22

Updated by Shweta about 1 month ago

Hi Mariusz

Can we get a new jboss patch in which jboss eap and tibco ems queue over ssl has been tested and getting connected successfully . We are not able to connect to tibco ems queue over ssl. Please assist us with the newest patch immediately.

Thanks,
Shweta Hegade,IPAM2

#23

Updated by Shweta about 1 month ago

Hi Mariusz,

Could you please let us know how to enable SSL TRACE in standalone-full.xml.

Please provide the update at the earliest.

Thanks,
Shweta Hegade,IPAM2

#24

Updated by Mariusz about 1 month ago

We are still waiting for reply from Red Hat.

#25

Updated by Mariusz about 1 month ago

  • Priority changed from High to Normal

Reply to comment #22:

 My name is Varsha and I am working on this case in absence of Tom. 

- What kind of authentication they were expecting ? Is it one way SSL or two way SSL ? 

There is no any patch to configure the SSL. We need to pass the certificate and password via JAVA options to make the connection secure. Those are configuration level changes. 

    JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore=/path/to/redhat.jks -Djavax.net.ssl.trustStorePassword=redhat

Similar kind of certificate and password you need to pass at Tibco level. 

Regards,
Varsha

Reply to comment #23

Hi,

Thank you for contacting Red Hat global support services.

You can add below java option to enable ssl debug logs in $JBOSS_HOME/bin/standalone.conf file (standalone mode). See article [1].
---
JAVA_OPTS="$JAVA_OPTS -Djavax.net.debug=ssl,handshake" 
---

Thank you.

Regards,
Ashish

[1] https://access.redhat.com/solutions/49082

I am changing the priority to normal as these issues are actually setup questions. For more information about Issues Severity visit: https://support.midvision.com/

Regards,
Mariusz Chwalek

#26

Updated by Shweta about 1 month ago

Hi Varsha,

We have made the configuration for SSL connection in our standalone.conf as suggested as below but still we are getting error.

JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore=/opt/jboss-eap-7.2/standalone/configuration/ipamrpaws.keystore -Djavax.net.ssl.trustStorePassword=*******

Find the attached error log. Please provide the update at the earliest.

#27

Updated by Shweta about 1 month ago

Hi Varsha,

We got below response from EMSBB team :

"There must be some parameter with joss framework that should autenticate the root and intermediate as a chain . At emsbb we allow one-way or two-way based on client presenting their certs .. if they present we autenticate again only the chain"

They allow one-way or two-way based on client presenting their certtificates. Currently at our end we are using two-way SSL configuration and MASSL is enabled.

Please let us know if any other configuration is required at our end .

Thanks,
Shweta Hegade,IPAM2

#28

Updated by Mariusz about 1 month ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#29

Updated by Shweta about 1 month ago

Hi Team,

EMSBB team is facing below error at their end after configurating above JVM argument (JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore=/opt/jboss-eap-7.2/standalone/configuration/ipamrpaws.keystore -Djavax.net.ssl.trustStorePassword=*******) at our end:

error at emsbb end:
2020-12-09 18:41:00.581 SSL handshake failed: ret=-1, reason=tlsv1 alert internal error
2020-12-09 18:41:00.589 [OpenSSL Error]: file=ossl.c, line=1393
2020-12-09 18:41:00.589 140356484171520:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_s3.c:1536:SSL alert number 80
2020-12-09 18:41:13.064 SSL handshake failed: ret=-1, reason=wrong version number
2020-12-09 18:41:13.080 [OpenSSL Error]: file=ossl.c, line=1393
2020-12-09 18:41:13.080 140356484171520:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:ssl/record/ssl3_record.c:332:

Kindly suggest an approach to proceed with this error resolution.

@Mariusz , TIBCO team has requested to have a call with RedHat Support team , IPAM2 team and EMSBB team to resolve this issue efficiently so please could you please ask RedHat Support team for their availability so that we can have this discussion on a call.

Please let us know about their availability as this issue has been escalated at our end and considered as a blocker.

Please provide an update at the earliest.

Thanks,
IPAM2 Application Team

#30

Updated by Mariusz about 1 month ago

The ticket will be updated as soon as we get more information from the RedHat Support team.

#31

Updated by Mariusz about 1 month ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Reply from Red Hat:

Unfortunately, Tom is out of office currently and he will be back in one week time.

Attached "tibco_ssl_error_2020-12-09.txt" showed this error at the end of the log file:
java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty

and I found following KCS articles relating to the error:
https://access.redhat.com/solutions/189763
https://access.redhat.com/solutions/1420563
https://access.redhat.com/solutions/4331001

You might want to take a look at them and hope they help.

In addition, since the issue involves Tibco SSL connection configuration. I wonder that have you contacted with Tibco support team about the SSL configuration and related errors?

Best Regards,
/Joe

#32

Updated by Shweta about 1 month ago

Hi Joe,
As suggested in the article https://access.redhat.com/solutions/189763 , we have configured system property javax.net.ssl.trustStore and javax.net.ssl.trustStorePassword in standalone-full.xml as below :

<property name="javax.net.ssl.trustStorePassword" value="*****"/>
<property name="javax.net.ssl.trustStore" value="${jboss.server.config.dir}/ipamrpaws.keystore"/>

Received below response after running this command : jinfo -sysprops ${JAVA_PID} | grep javax.net.ssl.trustStore :
[Fri 11/12 01:26 AM] root@ipam2-test-rpa:~# jinfo -sysprops ${JAVA_PID}
Usage:
jinfo <option> <pid>
(to connect to a running process)

where <option> is one of:
flag <name> to print the value of the named VM flag
-flag [+|
]<name> to enable or disable the named VM flag
-flag <name>=<value> to set the named VM flag to the given value
-flags to print VM flags
-sysprops to print Java system properties
<no option> to print both VM flags and system properties
-? | -h | --help | -help to print this help message

We have verified that keystore we configured in the property javax.net.ssl.trustStore consists of valid CA certs.
We also have configured above ssl property in java_opts in standalone.conf.

Currently we are using java-11-openjdk-11.0.8.10-0.el7_8.x86_64 in jboss. its cacerts ( path : /usr/lib/jvm/java-11-openjdk-11.0.8.10-0.el7_8.x86_64/lib/security/cacerts ) does not have have root cert of our server which is there in the trustStore we configured. Will that impact SSL connection ?

find the attached complete server logs. Getting the same error : java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty

Thanks,
IPAM2

#33

Updated by Mariusz about 1 month ago

Reply from Red Hat:

Hi

I'm looking at this whilst Tom is away; to be honest I haven't had chance to read the whole history yet, and I apologise if I'm missing something.

It seems to me that there are various things that need to be checked here.

1. I think my colleagues were asking you to run "jinfo" so that we could check that EAP is setting the correct system properties on the JVM. It's possible that there's something odd about the EAP configuration, that is causing it to set incorrect values of javax.net.ssl.trustStore, etc. I don't know why jinfo didn't work for you -- did you just run it incorrectly? I think when my colleagues used the expression ${JAVA_PID}, they were expecting you to substitute the actual process ID of the running server. The output from jinfo should be a list of properties, like this:

java.vm.name = OpenJDK 64-Bit Server VM
hawtio.realm = hawtio-domain
jboss.server.base.dir = /home/kevin/lib/jboss-eap-7.1/standalone
...

It might be hundreds of lines long. We really only need to see the javax.net.ssl.xxx entries, but it won't hurt to send the whole thing.

2. In answer to your question about "cacerts" -- if you specify a trust store on the Java command line (via EAP, in your case), then the JVM's cacerts is ignored. The value of javax.net.ssl.trustStore is an alternative to the system cacerts file. It is therefore vital that the trust store you specify contains a complete certificate chain, that will allow the server's certificate to be validated. That is, all certificates related to the server certificate must be in the trust store you specify. We have ways to check that, but I don't think we're yet in a position to interpret the results.

3. The error message "no trust anchors found" generally means that the specified trust store is corrupt, or just empty. I really need to know in detail how you generated the file "ipamrpaws.keystore". The name "keystore" here is a little suspicious, but it might just be the way you name things. The trust store is not usually a key store, it is a public certificate store. It's annoying that Java uses the term "keystore" for both things, so it's difficult to tell if a mistake was made.

Because the trust store contains (if correctly set up), only public-key certificates from your server, it should be safe for you to send it to us to look at (we'd need the password as well). If you can't send it, we do need to know in detail how you generated it.

Best wishes
Kevin

#34

Updated by Shweta about 1 month ago

Hi Kevin ,

I am not able to run the jiinfo command correctly but while starting jboss server we can see below system properties has been configured:
javax.net.debug = all
javax.net.ssl.trustStore = /opt/jboss-eap-7.2/standalone/configuration/clients.jks
javax.net.ssl.trustStorePassword = <redacted>

Configuration in standlone.conf :
  1. SSL connection for tibco
    JAVA_OPTS="$JAVA_OPTS -Djavax.net.ssl.trustStore=/opt/jboss-eap-7.2/standalone/configuration/clients.jks -Djavax.net.ssl.trustStorePassword=******"

#Enable SSL handshake debug logs
#JAVA_OPTS="$JAVA_OPTS -Djavax.net.debug=ssl,handshake"
JAVA_OPTS="$JAVA_OPTS -Djavax.net.debug=all"

Find the attached file consisting of all the system properties entries.
Let me know if this helps.

Thanks,
Shweta Hegade

#35

Updated by Mariusz about 1 month ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#36

Updated by Shweta about 1 month ago

Hi Kevin,

Currently we have configured client.jks in the javax.net.ssl.trustStore property. The same truststore is configured in standalone-full.xml which consists of valid certificate chain issued from PKI :

Configuration in standalone-full.xml :
<security-realm name="ManagementRealmHTTPS">
<server-identities>
<ssl>
<engine enabled-protocols="TLSv1.2"/>
<keystore path="ipamrpaws.keystore" relative-to="jboss.server.config.dir" keystore-password="*****" alias="server"/>
</ssl>
</server-identities>
<authentication>
<truststore path="/opt/jboss-eap-7.2/standalone/configuration/clients.jks" keystore-password="******"/>
<properties path="https-mgmt-users.properties" relative-to="jboss.server.config.dir"/>
</authentication>
<authorization>
<properties path="https-mgmt-groups.properties" relative-to="jboss.server.config.dir"/>
</authorization>
</security-realm>

Let me know if any other information is required.

Thanks,
Shweta Hegade

#37

Updated by Shweta about 1 month ago

Hi Team,

We are able to connect to tibco ems request queue over ssl . We have added below properties in the message driven bean :
"
@MessageDriven(name = "SearchSubnetMDB", activationConfig = {
@ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
@ActivationConfigProperty(propertyName = "destination", propertyValue = "Telstra.IPAM.SearchSubnet.Request"),
@ActivationConfigProperty(propertyName = "jndiParameters", propertyValue = "java.naming.security.principal=n109978;java.naming.security.credentials=*******;java.naming.factory.initial=com.tibco.tibjms.naming.TibjmsInitialContextFactory;java.naming.provider.url=tibjmsnaming://cly-prim1-dev.emsbb.telstra.com:7660,tibjmsnaming://cly-prim1-dev.emsbb.telstra.com:8888,tibjmsnaming://cly-prim2-dev.emsbb.telstra.com:8889,tibjmsnaming://cly-prim2-dev.emsbb.telstra.com:7660;com.tibco.tibjms.naming.security_protocol=ssl;com.tibco.tibjms.naming.ssl_enable_verify_host=false;com.tibco.tibjms.naming.ssl_enable_verify_hostname=false"),
@ActivationConfigProperty(propertyName = "connectionFactory", propertyValue = "n109978_IPAM2_QCF"),
@ActivationConfigProperty(propertyName = "user", propertyValue = "n109978"),
@ActivationConfigProperty(propertyName = "password", propertyValue = "*******"),
@ActivationConfigProperty(propertyName = "maxSession", propertyValue = "15"),
@ActivationConfigProperty(propertyName = "reconnectAttempts", propertyValue = "60")})
@ResourceAdapter("genericjms-xa.rar")"

These configurations we have added for request queue. Could you please suggest how to make configuratio for response queue in the jboss standalon-full.xml and at the code level.

Let us know if you require any other information from our end.

Thanks,
Shweta Hegade

#38

Updated by Mariusz about 1 month ago

Hi,
Your comment has been shared with Red Hat Support team.
We are still waiting for more information from them.
Thanks,
Mariusz Chwalek

#39

Updated by Mariusz about 1 month ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Reply from Red Hat:

Hi

My concern is this statement:

"The same truststore is configured in standalone-full.xml which consists of valid certificate chain issued from PKI"

How do you know this trust store is valid? The error message "the trustAnchors parameter must be non-empty" sounds very complicated but, in fact, almost always has a simple cause: the trust store is either empty, corrupt, or cannot be opened because of inadequate filesystem permissions.

What happens when you do "keytool -list -keystore [file]" on client.jks?

The simplest way to check this out would be for you to send us the trust store and password. If you can't do that because of security concerns, then that itself points to a problem. There should only be public-key certificates in the trust store, so it should be perfectly safe to share it. If it isn't safe to share it, that suggests that there's something in there, other than public key certificates, which would be a problem (if you see what I mean).

I'm not really sure what else to suggest -- all the evidence that I've seen suggests that your trust store is either not where EAP thinks it is, or doesn't contain any usable public-key certificates. I guess it could also have the wrong file permissions.

Best wishes
Kevin

PS. Your last message appeared whilst I was writing this. I'm afraid I don't understand how that new information fits into our previous discussions.

#40

Updated by Shweta about 1 month ago

Hi Team,

We are able to connect to tibco ems queue over ssl without MASSL configuration. We have added below jndiProperties at application code level :

@ActivationConfigProperty(propertyName = "jndiParameters", propertyValue = "java.naming.security.principal=n109978;java.naming.security.credentials=******;java.naming.factory.initial=com.tibco.tibjms.naming.TibjmsInitialContextFactory;java.naming.provider.url=tibjmsnaming://cly-prim1-dev.emsbb.telstra.com:7660,tibjmsnaming://cly-prim1-dev.emsbb.telstra.com:8888,tibjmsnaming://cly-prim2-dev.emsbb.telstra.com:8889,tibjmsnaming://cly-prim2-dev.emsbb.telstra.com:7660;com.tibco.tibjms.naming.security_protocol=ssl;com.tibco.tibjms.naming.ssl_enable_verify_host=true;com.tibco.tibjms.naming.ssl_enable_verify_hostname=false;
com.tibco.tibjms.naming.ssl_trusted_certs=${ssl.trusted_certs};com.tibco.tibjms.naming.ssl_identity=${ssl.identity};com.tibco.tibjms.naming.ssl_password=******;
com.tibco.tibjms.naming.ssl_private_key=${ssl.private_key} "),

Above variables has been added to jboss system properties like below and verified that jboss is able to read above properties :
<system-properties>
<property name="ssl.identity" value="${jboss.server.config.dir}/ipam2-test-rpa_identity.pem"/>
<property name="ssl.trusted_certs" value="${jboss.server.config.dir}/trusted_certs_tibco_ems.pem"/>
<property name="ssl.private_key" value="${jboss.server.config.dir}/ipamrpaws_pem_p12_private.key.pem"/>
</system-properties>

But we are getting below error while trying to establish the connection with tibco ems queue while MASSL is enabled:

javax.naming.AuthenticationException: Not permitted: Failed to connect to any server at: ssl://cly-prim1-dev.emsbb.telstra.com:7660, ssl://cly-prim1-dev.emsbb.telstra.com:8888, ssl://cly-prim2-dev.emsbb.telstra.com:8889, ssl://cly-prim2-dev.emsbb.telstra.com:7660 [Error: 'Error in params trusted certificate (1 of 2) ('No trusted certificates found')': url that returned this exception = ssl://cly-prim1-dev.emsbb.telstra.com:7660 ] [Root exception is javax.jms.JMSSecurityException: Failed to connect to any server at: ssl://cly-prim1-dev.emsbb.telstra.com:7660, ssl://cly-prim1-dev.emsbb.telstra.com:8888, ssl://cly-prim2-dev.emsbb.telstra.com:8889, ssl://cly-prim2-dev.emsbb.telstra.com:7660 [Error: 'Error in params trusted certificate (1 of 2) ('No trusted certificates found')': url that returned this exception = ssl://cly-prim1-dev.emsbb.telstra.com:7660 ]]
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:673)
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:494)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)

Could you please suggest how to resolve this issue. PFA the detailed error log file.

Thanks,
Shweta Hegade,IPAM2 Application team.

#41

Updated by Mariusz about 1 month ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#42

Updated by Mariusz about 1 month ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Reply from Red Hat:

The error that you see
~~
javax.naming.AuthenticationException: Not permitted: Failed to connect to any server at: ssl://cly-prim1-dev.emsbb.telstra.com:7660, ssl://cly-prim1-dev.emsbb.telstra.com:8888, ssl://cly-prim2-dev.emsbb.telstra.com:8889, ssl://cly-prim2-dev.emsbb.telstra.com:7660 [Error: 'Error in params trusted certificate (1 of 2) ('No trusted certificates found')': url that returned this exception = ssl://cly-prim1-dev.emsbb.telstra.com:7660 ] [Root exception is javax.jms.JMSSecurityException: Failed to connect to any server at: ssl://cly-prim1-dev.emsbb.telstra.com:7660, ssl://cly-prim1-dev.emsbb.telstra.com:8888, ssl://cly-prim2-dev.emsbb.telstra.com:8889, ssl://cly-prim2-dev.emsbb.telstra.com:7660 [Error: 'Error in params trusted certificate (1 of 2) ('No trusted certificates found')': url that returned this exception = ssl://cly-prim1-dev.emsbb.telstra.com:7660 ]]
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:673)
at //com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:494)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
~~

is thrown by TIBCO run time because the authentication has failed. It seems that either your password or certificate is incorrect. It could be that the your trust store is empty, see "('No trusted certificates found')'"

Are you able to connect to the broker over SSL outside the JBoss EAP?

Best regards - Tom Ross

#43

Updated by Mariusz about 1 month ago

From Red Hat:

Could tell what is the status of this case?

Best regards - Tom Ross

#44

Updated by Shweta 19 days ago

Hi Team,

We are still not able to connect to tibco ems queue over MASSL enabled.
Please find the attached standalone-full.xml .

Thanks,
Shweta Hegade,IPAM2

#45

Updated by Mariusz 19 days ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Shweta to Red Hat Support
#46

Updated by Mariusz 13 days ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Red Hat Support to Shweta

Reply from Red Hat:
You configuration seems to be OK though since we never tested JBoss EAP Generic RA with TIBCO and SSL I can't comment on the SSL part of your configuration.
Looking at the naming configuration in your standalone*.xml I'd add

 <property name="org.jboss.as.naming.lookup.by.string" value="true"/>

So it would look like
<external-context name="java:global/tibco" module="org.jboss.genericjms.provider" class="javax.naming.InitialContext">
                    <environment>
                        <property name="java.naming.factory.initial" value="com.tibco.tibjms.naming.TibjmsInitialContextFactory"/>
                        <property name="java.naming.factory.url.pkgs" value="com.tibco.tibjms.naming"/>
                        <property name="java.naming.provider.url" value="tcp://${tibco.host}:${tibco.port}"/>
                        <property name="java.naming.security.principal" value="${tibco.user}"/>
                        <property name="java.naming.security.credentials" value="${tibco.password}"/>
                        <property name="org.jboss.as.naming.lookup.by.string" value="true"/>
                    </environment>
                </external-context>

Have you been in contact with TIBCO concerning SSL setup?

Best regards - Tom Ross

#47

Updated by Mariusz 4 days ago

Reply from RedHat Support team regarding both #3689 and #3705:

Hi,

As I said at the beginning if you let us know how to configure TIBCO with SSL  we could try to see if it is possible to use Generic RA with TIBCO and SSL. At the stage we have never tried to set up JBoss EAP server with Generic JMS RA with TIBCO and SSL so we have no idea if it possible or if it works.

I would start with trying to see if they can use JBoss EAP with TIBCO without SSL and then progress from there.

Best regards - Tom Ross

Also available in: Atom PDF