How to monitor a Java EE DataSource

Introduction

FlexyPool is an open-source framework that can monitor a DataSource connection usage. This tool come out of necessity, since we previously lacked support for provisioning connection pools.

FlexyPool was initially designed for stand-alone environments and the DataSource proxy configuration was done programmatically. Using Spring bean aliases, we could even substitute an already configured DataSource with the FlexyPool Metrics-aware proxy alternative.

Java EE support

Recently, I’ve been asked about supporting Java EE environments and in the true open-source spirit, I accepted the challenge. Supporting a managed environment is tricky because the DataSource is totally decoupled from the application-logic and made available through a JNDI lookup.

One drawback is that we can’t use automatic pool sizing strategies, since most Application Servers return a custom DataSource implementation (which is closely integrated with their in-house JTA transaction manager solution), that doesn’t offer access to reading/writing the connection pool size.

While the DataSource might not be adjustable, we can at least monitor the connection usage and that’s enough reason to support Java EE environments too.

Adding declarative configuration

Because we operate in a managed environment, we can no longer configure the DataSource programmatically, so we need to use the declarative configuration support.

By default, FlexyPool looks for the flexy-pool.properties file in the current Class-path. The location can be customized using the flexy.pool.properties.path System property , which can be a:

  • URL (e.g. file:/D:/wrk/vladmihalcea/flexy-pool/flexy-pool-core/target/test-classes/flexy-pool.properties)
  • File system path (e.g. D:\wrk\vladmihalcea\flexy-pool\flexy-pool-core\target\test-classes\flexy-pool.properties)
  • Class-path nested path (e.g. nested/fp.properties)

The properties file may contain the following configuration options:

Parameter name Description
flexy.pool.data.source.unique.name

Each FlexyPool instance requires a unique name so that JMX domains won’t clash

flexy.pool.data.source.jndi.name

The JNDI DataSource location

flexy.pool.data.source.jndi.lazy.lookup

Whether to lookup the DataSource lazily (useful when the target DataSource is not available when the FlexyPoolDataSource is instantiated)

flexy.pool.data.source.class.name

The DataSource can be instantiated at Runtime using this Class name

flexy.pool.data.source.property.*

If the DataSource is instantiated at Runtime, each flexy.pool.data.source.property.${java-bean-property} will set the java-bean-property of the newly instantiated DataSource (e.g. flexy.pool.data.source.property.user=sa)

flexy.pool.adapter.factory

Specifies the PoolAdaptorFactory, in case the DataSource supports dynamic sizing. By default it uses the generic DataSourcePoolAdapter which doesn’t support auto-scaling

flexy.pool.metrics.factory

Specifies the MetricsFactory used for creating Metrics

flexy.pool.metrics.reporter.log.millis

Specifies the metrics log reported interval

flexy.pool.metrics.reporter.jmx.enable

Specifies if the jmx reporting should be enabled

flexy.pool.metrics.reporter.jmx.auto.start

Specifies if the jmx service should be auto-started (set this to true in Java EE environments)

flexy.pool.strategies.factory.resolver

Specifies a ConnectionAcquiringStrategyFactoryResolver class to be used for obtaining a list of ConnectionAcquiringStrategyFactory objects. This should be set only if the PoolAdaptor supports accessing the DataSource pool size.

Hibernate ConnectionProvider

Most Java EE applications already use JPA and for those who happen to be using Hibernate, we can make use of the hibernate.connection.provider_class configuration property for injecting our proxy DataSource.

Hibernate provides many built-in extension points and the connection management is totally configurable. By providing a custom ConnectionProvider we can substitute the original DataSource with the FlexyPool proxy.

All we have to do is adding the following property to our persistence.xml file:

<property name="hibernate.connection.provider_class"
          value="com.vladmihalcea.flexypool.adaptor.FlexyPoolHibernateConnectionProvider"/>

Behind the scenes, this provider will configure a FlexyPoolDataSource and use it whenever a new connection is requested:

private FlexyPoolDataSource<DataSource> flexyPoolDataSource;

@Override
public void configure(Map props) {
    super.configure(props);
    LOGGER.debug(
        "Hibernate switched to using FlexyPoolDataSource
    ");
    flexyPoolDataSource = new FlexyPoolDataSource<DataSource>(
        getDataSource()
    );
}

@Override
public Connection getConnection() throws SQLException {
    return flexyPoolDataSource.getConnection();
}

Instantiating the actual DataSource at runtime

If you’re not using Hibernate, you need to have the FlexyPoolDataSource ready before the EntityManagerFactory finishes bootstrapping:

<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
        http://java.sun.com/xml/ns/persistence
        http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">

    <persistence-unit name="persistenceUnit" transaction-type="JTA">

        <provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>

        <jta-data-source>java:global/jdbc/flexypool</jta-data-source>

        <properties>
            <property 
                name="hibernate.hbm2ddl.auto" 
                value="update"/>

            <property 
                name="hibernate.show_sql" 
                value="true"/>

            <property 
                name="hibernate.dialect" 
                value="org.hibernate.dialect.HSQLDialect"/>

            <property 
                name="hibernate.transaction.jta.platform" 
                value="org.hibernate.service.jta.platform.internal.SunOneJtaPlatform"/>
        </properties>
    </persistence-unit>
</persistence>

While in a production Java EE environment we use an Application server specific DataSource configuration, for simplicity sake, I’m going to configure the FlexyPooldataSource using the DataSourceDefinition annotation:

@DataSourceDefinition(
    name = "java:global/jdbc/flexypool",
    className = "com.vladmihalcea.flexypool.FlexyPoolDataSource")
@Stateless
public class FlexyPoolDataSourceConfiguration {}

We now need to pass the actual DataSource properties to FlexyPool and this is done through the flexy-pool.properties configuration file:

flexy.pool.data.source.unique.name=unique-name
flexy.pool.data.source.class.name=org.hsqldb.jdbc.JDBCDataSource
flexy.pool.data.source.property.user=sa
flexy.pool.data.source.property.password=
flexy.pool.data.source.property.url=jdbc:hsqldb:mem:test
flexy.pool.metrics.reporter.jmx.auto.start=true

The actual DataSource is going to be created by the FlexyPoolDataSource on start-up.

Locating the actual DataSource from JNDI

If the actual DataSource is already configured by the Application Server, we can instruct FlexyPool to fetch it from JNDI. Let’s say we have the following DataSource configuration:

@DataSourceDefinition(
	name = "java:global/jdbc/default",
	className = "org.hsqldb.jdbc.JDBCDataSource",
	url = "jdbc:hsqldb:mem:test",
	initialPoolSize = 3,
	maxPoolSize = 5
)
@Stateless
public class DefaultDataSourceConfiguration {}

To proxy the JNDI DataSource, we need to configure FlexyPool like this:

flexy.pool.data.source.unique.name=unique-name
flexy.pool.data.source.jndi.name=java:global/jdbc/default
flexy.pool.metrics.reporter.jmx.auto.start=true

The FlexyPoolDataSource is defined alongside the actual DataSource:

@DataSourceDefinition(
	name = "java:global/jdbc/flexypool",
	className = "com.vladmihalcea.flexypool.FlexyPoolDataSource")
@Stateless
public class FlexyPoolDataSourceConfiguration {}

The JPA will have to fetch the FlexyPoolDataSource instead of the actual one:

<jta-data-source>java:global/jdbc/flexypool</jta-data-source>

In TomEE, because the DataSourceDefinitions are not lazily instantiated, the actual DataSource might not be available in the JNDI registry when the FlexyPoolDataSource definition is processed.

For this, we need to instruct FlexyPool to dely the JNDI lookup until the DataSource is actually requested:

flexy.pool.data.source.jndi.lazy.lookup=true

If you enjoyed this article, I bet you are going to love my book as well.

Conclusion

The last time I used Java EE was in 2008, on a project that was using Java EE 1.4 with EJB 2.1. After 7 years of using Spring exclusively, I’m pleasantly surprised by the Java EE experience. Arquillian is definitely my favourite add-on, since integration testing is of paramount importance in enterprise applications. CDI is both easy and powerful and I’m glad the dependency injection got standardised.

But the best asset of the Java EE platform is the community itself. Java EE has very strong community, willing to give you a hand when in need. I’d like to thank Steve Millidge (Founder of Payara and C2B2) for giving me some great tips on designing the FlexyPool Java EE integration, Alex Soto, Antonio Goncalves, Markus Eisele and all the other Java EE members whom I had some very interesting conversations on Twitter.

Enter your email address to follow this blog and receive notifications of new posts by email.

Advertisements

12 thoughts on “How to monitor a Java EE DataSource

  1. With this flexy-pool.properties

    flexy.pool.data.source.unique.name=flexy-pool
    flexy.pool.data.source.jndi.name=java:comp/env/datasource/db
    flexy.pool.metrics.reporter.jmx.auto.start=true
    flexy.pool.metrics.reporter.log.millis=5000

    and

    dependency>
    com.vladmihalcea.flexy-pool
    flexy-tomcatcp
    1.2.1

    how I can see if flexy-pool is running ? I don’t know if it writes some log, or if I must configure some else .

    1. You can connect to the JMX server using the jconsole utility (you can find it in your $JAVA_HOME/bin folder). The logs are generates using slf4j-api, so you need an actual slf4j bridge (slf4j-logback or slf4j-log4j ) to get the logging working.

      1. Thanks. I want to see datasource usage in my logs I don’t want use JMX

        I use

        org.slf4j
        slf4j-api
        ${slf4j.version}

        ch.qos.logback
        logback-classic
        ${logback.version}

        on DEBUG level, I don’t see anything.
        I guess that I’m missing some setting in flexy-pool.properties
        The datasource is defined in my META-INF/context,xml and exported by JNDI

        Regards

      2. The Slf4jMetricReporter fires every 5 seconds, so you should have seen it if everything works properly. You can debug the FlexyPoolDataSource object initialization to see if it starts properly.

  2. Finally it works. I did a FlexyPoolDatasourceFactory which provides FlexyPoolDatasource ( a singleton created by Spring context )
    Regards

      1. So far I’m testing it . If we install it on a real environment , I’ll give to you feedbak about that. Regards

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s