Vlad Mihalcea

How to import CSV data into PostgreSQL

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

Many database servers support CSV data transfers and this post will show one way you can import CSV files to PostgreSQL.

SQL aggregation rocks!

My previous post demonstrated FlexyPool metrics capabilities and all connection related statistics were exported in CSV format.

When it comes to aggregation tabular data SQL is at its best. If your database engine supports SQL:2003 windows functions you should definitely make use of this great feature.

Read More

Professional connection pool sizing with FlexyPool

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

I previously wrote about the benefits of connection pooling and why monitoring it is of crucial importance. This post will demonstrate how FlexyPool can assist you in finding the right size for your connection pools.

Know your connection pool

The first step is to know your connection pool settings. My current application uses XA transactions, therefore we use Bitronix transaction manager, which comes with its own connection pooling solution.

Accord to the Bitronix connection pool documentation we need to use the following settings:

  • minPoolSize: the initial connection pool size
  • maxPoolSize: the maximum size the connection pool can grow to
  • maxIdleTime: the maximum time a connection can remain idle before being destroyed
  • acquisitionTimeout: the maximum time a connection request can wait before throwing a timeout. The default value of 30s is way too much for our QoS

Read More

FlexyPool, reactive connection pooling

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

When I started working on enterprise projects we were using J2EE and the pooling data source was provided by the application server.

Enterprise Application Server

Scaling up meant buying more powerful hardware to support the increasing request demand. The vertical scaling meant that for supporting more requests, we would have to increase the connection pool size accordingly.

Horizontal scaling

Our recent architectures shifted from scaling up to scaling out. So instead of having one big machine hosting all our enterprise services, we now have a distributed service network.

Enterprise Architecture

This has numerous advantages:

  • Each JVM is tuned according to the hosted service intrinsic behaviour. Web nodes employ the concurrent low pause collector, while batch services use the throughput collector
  • Deploying a batch service doesn’t affect the front services
  • If one service goes down it won’t affect the rest

Read More

The anatomy of Connection Pooling

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

All projects I’ve been working on have used database connection pooling and that’s for very good reasons. Sometimes we might forget why we are employing one design pattern or a particular technology, so it’s worth stepping back and reason on it. Every technology or technological decision has both upsides and downsides, and if you can’t see any drawback you need to wonder what you are missing.

The database connection life-cycle

Every database read or write operation requires a connection. So let’s see how database connection flow looks like:

Connection Lifecycle

The flow goes like this:

  1. The application data layer ask the DataSource for a database connection
  2. The DataSource will use the database Driver to open a database connection
  3. A database connection is created and a TCP socket is opened
  4. The application reads/writes to the database
  5. The connection is no longer required so it is closed
  6. The socket is closed

You can easily deduce that opening/closing connections are quite an expensive operation. PostgreSQL uses a separate OS process for every client connection, so a high rate of opening/closing connections is going to put a strain on your database management system.

The most obvious reasons for reusing a database connection would be:

  • reducing the application and database management system OS I/O overhead for creating/destroying a TCP connection
  • reducing JVM object garbage

Read More

Maven and Java multi-version modules

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

In this article, you are going to learn how to configure Maven to choose a specific Java version when the OS has multiple Java versions installed.

For instance, FlexyPool uses Java 8 for all modules, except for the flexy-pool-core-java9 module that needs to be built using Java 9.

Read More

MongoDB 2.6 is $out

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

MongoDB is evolving rapidly. The 2.2 version introduced the aggregation framework as an alternative to the Map-Reduce query model. Generating aggregated reports is a recurrent requirement for enterprise systems and MongoDB shines in this regard. If you’re new to it you might want to check this aggregation framework introduction or the performance tuning and the data modelling guides.

Read More

Effective learning techniques for software craftsmen

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Go in one ear and out the other

Programming languages, operating systems, SQL, NoSQL, web frameworks, Spring, Java EE, HTML, JavaScript, Agile methodologies, you name it. A developer must know a ridiculous amount of things to become effective. There’s no wonder many of us are struggling to keep pace with the ever-changing programming landscape.

When you’re a kid, doing stuff is the most natural way of learning, but then you go to school and you’re brainwashed into thinking that reading is the only way of studying.

Become an active learner

Ever since I started writing this blog I began to question my old ways of learning. I used to be a passive learner, reading books, articles or watching videos. But I realized this is not working the way it should, so I started looking for alternatives.

Edgar Dale‘s evaluated the most common learning techniques and came up with the famous Cone of experience. Basically, if you want to be a better developer you need to become an active learner.

Writing a blog

Until you start writing, you don’t really appreciate the actual effort put into coming up with a decent article. Writing down your findings not only helps the community, it helps you too. The writing process is going to teach you more about a given subject. You want to publish quality articles and the upcoming critique is going to be a good motivator.

Read More

The Builder pattern and the Spring framework

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

I like to make use of the builder pattern whenever an object has both mandatory and optional properties. But building objects is usually the Spring framework responsibility, so let’s see how you can employ it using both Java and XML-based Spring configurations.

Read More

Afraid of reopened issues?

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

Introduction

Reopened issues and developer feelings don’t mix well, a recurrent phenomenon I’ve seen on all projects I’ve worked on. Some might feel they’ve worked “in vain”, being reluctant to restart it all over again.

Reopened issues are bound to happen

There is a thin line between taking ownership of your current project and remaining professionally detached at all times. The only thing that matters is the value the customer gets for any given issue, even if it takes more steps than you previously anticipated. In software development, the change is the only thing that never changes, that’s why you’ll always have to deal with reopened issues. Reopening an issue is not necessarily a bad thing, as you’ll soon find out.

Read More

Choosing a leader like an agilist

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Hypersistence Optimizer is that tool!

The leader as a captain

I recently read Petri Kainulainen’s article on sharing leadership among team members and I am on the same wavelength in this regard, since the Agile methods emphasizes the importance of “motivated individuals, who should be trusted”.

While a team leader could be regarded as a reminiscence of the old rigid organization structures, I still see many benefits of having such a captain. When it comes to improving my people skills, I like to get inspired by other domains of activity that have been struggling with the very same challenges for decades. Think about the greatest sports teams, where all team members are stars, and yet they always have a captain for:

  • the captain is the role-model for every team member
  • he leads by example
  • he always mediates conflicts, knowing that prevention is always better than curing
  • he welcomes and facilitates new team members integration
  • he never loses his temper
  • in times of trouble, he is the voice of the team
  • he is constantly goal-oriented, making sure the team is on the right track

Read More

High-Performance SQL Online Workshop - 27th of May