Integration testing done right with Embedded MongoDB

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Wouldn’t that be just awesome?

Well, Hypersistence Optimizer is that tool! And it works with Spring Boot, Spring Framework, Jakarta EE, Java EE, Quarkus, or Play Framework.

So, enjoy spending your time on the things you love rather than fixing performance issues in your production system on a Saturday night!


Unit testing requires isolating individual components from their dependencies. Dependencies are replaced with mocks, which simulate certain use cases. This way, we can validate the in-test component behavior across various external context scenarios.

Web components can be unit tested using mock business logic services. Services can be tested against mock data access repositories. But the data access layer is not a good candidate for unit testing because database statements need to be validated against an actual running database system.

Integration testing database options

Ideally, our tests should run against a production-like database. But using a dedicated database server is not feasible, as we most likely have more than one developer to run such integration test-suites. To isolate concurrent test runs, each developer would require a dedicated database catalog. Adding a continuous integration tool makes matters worse since more tests would have to be run in parallel.

Lesson 1: We need a forked test-suite bound database

When a test suite runs, a database must be started and only made available to that particular test-suite instance. Basically we have the following options:

  • An in-memory embedded database
  • A temporary spawned database process

The fallacy of in-memory database testing

Java offers multiple in-memory relational database options to choose from:

Embedding an in-memory database is fast and each JVM can run its own isolated database. But we no longer test against the actual production-like database engine because our integration tests will validate the application behavior for a non-production database system.

Using an ORM tool may provide the false impression that all databases are equal, especially when all generated SQL code is SQL-92 compliant.

What’s good for the ORM tool database support may deprive you from using database specific querying features (Window Functions, Common Table Expressions, PIVOT).

So the integration testing in-memory database might not support such advanced queries. This can lead to reduced code coverage or to pushing developers to only use the common-yet-limited SQL querying features.

Even if your production database engine provides an in-memory variant, there may still be operational differences between the actual and the lightweight database versions.

Lesson 2: In-memory databases may give you the false impression that your code will also run on a production database

Spawning a production-like temporary database

Testing against the actual production database is much more valuable and that’s why I grew to appreciate this alternative.

When using MongoDB we can choose the embedded mongo plugin. This open-source project creates an external database process that can be bound to the current test-suite life-cycle.

If you’re using Maven, you can take advantage of the embedmongo-maven-plugin:


When running the plugin, the following actions are taken:

  1. A MongoDB pack is downloaded

    [INFO] --- embedmongo-maven-plugin:0.1.12:start (start) @ mongodb-facts ---
    Download Version{2.6.1}:Windows:B64 START
    Download Version{2.6.1}:Windows:B64 DownloadSize: 135999092
    Download Version{2.6.1}:Windows:B64 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% 11% 12% 13% 14% 15% 16% 17% 18% 19% 20% 21% 22% 23% 24% 25% 26% 27% 28% 29% 30% 31% 32% 33% 34% 35% 36% 37% 38% 39% 40% 41% 42% 43% 44% 45% 46% 47% 48% 49% 50% 51% 52% 53% 54% 55% 56% 57% 58% 59% 60% 61% 62% 63% 64% 65% 66% 67% 68% 69% 70% 71% 72% 73% 74% 75% 76% 77% 78% 79% 80% 81% 82% 83% 84% 85% 86% 87% 88% 89% 90% 91% 92% 93% 94% 95% 96% 97% 98% 99% 100% Download Version{2.6.1}:Windows:B64 downloaded with 3320kb/s
    Download Version{2.6.1}:Windows:B64 DONE
  2. Upon starting a new test suite, the MongoDB pack is unzipped under a unique location in the OS temp folder

    Extract C:\Users\vlad\.embedmongo\win32\ START
    Extract C:\Users\vlad\.embedmongo\win32\ DONE
  3. The embedded MongoDB instance is started.

    [mongod output]note: noprealloc may hurt performance in many applications
    [mongod output] 2014-10-09T23:25:16.889+0300 [DataFileSync] warning: --syncdelay 0 is not recommended and can have strange performance
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] MongoDB starting : pid=2384 port=51567 dbpath=D:\wrk\vladmihalcea\mongodb-facts\target\mongotest 64-bit host=VLAD
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] db version v2.6.1
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] allocator: system
    [mongod output] 2014-10-09T23:25:16.891+0300 [initandlisten] options: { net: { bindIp: "", http: { enabled: false }, port: 51567 }, security: { authorization: "disabled" }, storage: { dbPath: "D:\wrk\vladmihalcea\mongodb-facts\target\mongotest", journal: { enabled: false }, preallocDataFiles: false, smallFiles: true, syncPeriodSecs: 0.0 } }
    [mongod output] 2014-10-09T23:25:17.179+0300 [FileAllocator] allocating new datafile D:\wrk\vladmihalcea\mongodb-facts\target\mongotest\local.ns, filling with zeroes...
    [mongod output] 2014-10-09T23:25:17.179+0300 [FileAllocator] creating directory D:\wrk\vladmihalcea\mongodb-facts\target\mongotest\_tmp
    [mongod output] 2014-10-09T23:25:17.240+0300 [FileAllocator] done allocating datafile D:\wrk\vladmihalcea\mongodb-facts\target\mongotest\local.ns, size: 16MB,  took 0.059 secs
    [mongod output] 2014-10-09T23:25:17.240+0300 [FileAllocator] allocating new datafile D:\wrk\vladmihalcea\mongodb-facts\target\mongotest\local.0, filling with zeroes...
    [mongod output] 2014-10-09T23:25:17.262+0300 [FileAllocator] done allocating datafile D:\wrk\vladmihalcea\mongodb-facts\target\mongotest\local.0, size: 16MB,  took 0.021 secs
    [mongod output] 2014-10-09T23:25:17.262+0300 [initandlisten] build index on: local.startup_log properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.startup_log" }
    [mongod output] 2014-10-09T23:25:17.262+0300 [initandlisten]     added index to empty collection
    [mongod output] 2014-10-09T23:25:17.263+0300 [initandlisten] waiting for connections on port 51567
    [mongod output] Oct 09, 2014 11:25:17 PM MongodExecutable start
    INFO: de.flapdoodle.embed.mongo.config.MongodConfigBuilder$ImmutableMongodConfig@26b3719c
  4. For the life-time of the current test-suite you can see the embedded-mongo process:

    C:\Users\vlad>netstat -ano | findstr 51567
      TCP              LISTENING       8500
    C:\Users\vlad>TASKLIST /FI "PID eq 8500"
    Image Name                     PID Session Name        Session#    Mem Usage
    ========================= ======== ================ =========== ============
    extract-0eecee01-117b-4d2     8500 RDP-Tcp#0                  1     44,532 K  


  5. When the test-suite is finished the embeded-mongo is stopped

    [INFO] --- embedmongo-maven-plugin:0.1.12:stop (stop) @ mongodb-facts ---
    2014-10-09T23:25:21.187+0300 [initandlisten] connection accepted from #11 (1 connection now open)
    [mongod output] 2014-10-09T23:25:21.189+0300 [conn11] terminating, shutdown command received
    [mongod output] 2014-10-09T23:25:21.189+0300 [conn11] dbexit: shutdown called
    [mongod output] 2014-10-09T23:25:21.189+0300 [conn11] shutdown: going to close listening sockets...
    [mongod output] 2014-10-09T23:25:21.189+0300 [conn11] closing listening socket: 520
    [mongod output] 2014-10-09T23:25:21.189+0300 [conn11] shutdown: going to flush diaglog...
    [mongod output] 2014-10-09T23:25:21.189+0300 [conn11] shutdown: going to close sockets...
    [mongod output] 2014-10-09T23:25:21.190+0300 [conn11] shutdown: waiting for fs preallocator...
    [mongod output] 2014-10-09T23:25:21.190+0300 [conn11] shutdown: closing all files...
    [mongod output] 2014-10-09T23:25:21.191+0300 [conn11] closeAllFiles() finished
    [mongod output] 2014-10-09T23:25:21.191+0300 [conn11] shutdown: removing fs lock...
    [mongod output] 2014-10-09T23:25:21.191+0300 [conn11] dbexit: really exiting now
    [mongod output] Oct 09, 2014 11:25:21 PM de.flapdoodle.embed.process.runtime.ProcessControl stopOrDestroyProcess


The embed-mongo plugin is nowhere slower than any in-memory relation database systems. It makes me wonder why there isn’t such an option for open-source RDBMS (e.g. PostgreSQL). This is a great open-source project idea and maybe Flapdoodle OSS will offer support for relational databases too.

Transactions and Concurrency Control eBook

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.