Professional connection pool sizing with FlexyPool

Introduction

I previously wrote about the benefits of connection pooling and why monitoring it is of crucial importance. This post will demonstrate how FlexyPool can assist you in finding the right size for your connection pools.

Know your connection pool

The first step is to know your connection pool settings. My current application uses XA transactions, therefore we use Bitronix transaction manager, which comes with its own connection pooling solution.

Accord to the Bitronix connection pool documentation we need to use the following settings:

  • minPoolSize: the initial connection pool size
  • maxPoolSize: the maximum size the connection pool can grow to
  • maxIdleTime: the maximum time a connection can remain idle before being destroyed
  • acquisitionTimeout: the maximum time a connection request can wait before throwing a timeout. The default value of 30s is way too much for our QoS

Continue reading “Professional connection pool sizing with FlexyPool”

Advertisements

FlexyPool, reactive connection pooling

Introduction

When I started working on enterprise projects we were using J2EE and the pooling data source was provided by the application server.

EnterpriseApplicationServer

Scaling up meant buying more powerful hardware to support the increasing request demand. The vertical scaling meant that for supporting more requests, we would have to increase the connection pool size accordingly.

Horizontal scaling

Our recent architectures shifted from scaling up to scaling out. So instead of having one big machine hosting all our enterprise services, we now have a distributed service network.

EnterpriseArchitecture

This has numerous advantages:

  • Each JVM is tuned according to the hosted service intrinsic behaviour. Web nodes employ the concurrent low pause collector, while batch services use the throughput collector
  • Deploying a batch service doesn’t affect the front services
  • If one service goes down it won’t affect the rest

Continue reading “FlexyPool, reactive connection pooling”

The anatomy of Connection Pooling

Introduction

All projects I’ve been working on have used database connection pooling and that’s for very good reasons. Sometimes we might forget why we are employing one design pattern or a particular technology, so it’s worth stepping back and reason on it. Every technology or technological decision has both upsides and downsides, and if you can’t see any drawback you need to wonder what you are missing.

The database connection life-cycle

Every database read or write operation requires a connection. So let’s see how database connection flow looks like:

Image

The flow goes like this:

  1. The application data layer ask the DataSource for a database connection
  2. The DataSource will use the database Driver to open a database connection
  3. A database connection is created and a TCP socket is opened
  4. The application reads/writes to the database
  5. The connection is no longer required so it is closed
  6. The socket is closed

You can easily deduce that opening/closing connections are quite an expensive operation. PostgreSQL uses a separate OS process for every client connection, so a high rate of opening/closing connections is going to put a strain on your database management system.

The most obvious reasons for reusing a database connection would be:

  • reducing the application and database management system OS I/O overhead for creating/destroying a TCP connection
  • reducing JVM object garbage

Continue reading “The anatomy of Connection Pooling”

Maven and Java multi-version modules

Introduction

Usually, a project has a minimum Java version requirement and that applies to all of its modules. But every rule has its exceptions, as recently I stumbled on the following issue.

One open source project of mine mandates Java 1.6 for most of its modules, except one requiring the 1.7 version.

This happens when integrating external libraries having different Java requirements than your own project.

Because that one module integrates the DBCP2 framework (supporting at least Java 1.7), I need to instruct Maven to use two different Java compilers.

Continue reading “Maven and Java multi-version modules”

MongoDB 2.6 is $out

Introduction

MongoDB is evolving rapidly. The 2.2 version introduced the aggregation framework as an alternative to the Map-Reduce query model. Generating aggregated reports is a recurrent requirement for enterprise systems and MongoDB shines in this regard. If you’re new to it you might want to check this aggregation framework introduction or the performance tuning and the data modelling guides.

Continue reading “MongoDB 2.6 is $out”