How to fetch entities multiple levels deep with Hibernate

Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Wouldn’t that be just awesome?

Well, Hypersistence Optimizer is that tool! And it works with Spring Boot, Spring Framework, Jakarta EE, Java EE, Quarkus, or Play Framework.

So, enjoy spending your time on the things you love rather than fixing performance issues in your production system on a Saturday night!

Introduction

It’s quite common to retrieve a root entity along with its child associations on multiple levels.

In our example, we need to load a Forest with its Trees and Branches and Leaves, and we will try to see have Hibernate behaves for three collection types: Sets, Indexed Lists, and Bags.

Domain Model

This is how our class hierarchy looks like:

Image

Too many joins

Using Sets and Indexed Lists is straightforward since we can load all entities by running the following JPA-QL query:

Forest f = entityManager
.createQuery(
    "select f " +
    "from Forest f " +
    "join fetch f.trees t " +
    "join fetch t.branches b " +
    "join fetch b.leaves l ", Forest.class)
.getSingleResult();

and the executed SQL query is:

SELECT forest0_.id        AS id1_7_0_,
       trees1_.id         AS id1_18_1_,
       branches2_.id      AS id1_4_2_,
       leaves3_.id        AS id1_10_3_,
       trees1_.forest_fk  AS forest_f3_18_1_,
       trees1_.index      AS index2_18_1_,
       trees1_.forest_fk  AS forest_f3_7_0__,
       trees1_.id         AS id1_18_0__,
       trees1_.index      AS index2_0__,
       branches2_.index   AS index2_4_2_,
       branches2_.tree_fk AS tree_fk3_4_2_,
       branches2_.tree_fk AS tree_fk3_18_1__,
       branches2_.id      AS id1_4_1__,
       branches2_.index   AS index2_1__,
       leaves3_.branch_fk AS branch_f3_10_3_,
       leaves3_.index     AS index2_10_3_,
       leaves3_.branch_fk AS branch_f3_4_2__,
       leaves3_.id        AS id1_10_2__,
       leaves3_.index     AS index2_2__
FROM   forest forest0_
INNER JOIN tree trees1_ ON forest0_.id = trees1_.forest_fk
INNER JOIN branch branches2_ ON trees1_.id = branches2_.tree_fk
INNER JOIN leaf leaves3_ ON branches2_.id = leaves3_.branch_fk

But when our children associations are mapped as Bags, the same JPQL query throws a MultipleBagFetchException.

If you want to learn the best way to fix the MultipleBagFetchException, then check out this article.

Too many queries

In case you can’t alter your mappings (replacing the Bags with Sets or Indexed Lists) you might be tempted to try the something like:

BagForest forest = entityManager.find(BagForest.class, forestId);

for (BagTree tree : forest.getTrees()) {
    for (BagBranch branch : tree.getBranches()) {
        branch.getLeaves().size();      
    }
}

But this is inefficient generating a plethora of SQL queries:

select trees0_.forest_id as forest_i3_1_1_, trees0_.id as id1_3_1_, trees0_.id as id1_3_0_, trees0_.forest_id as forest_i3_3_0_, trees0_.index as index2_3_0_ from BagTree trees0_ where trees0_.forest_id=? 
             
select branches0_.tree_id as tree_id3_3_1_, branches0_.id as id1_0_1_, branches0_.id as id1_0_0_, branches0_.index as index2_0_0_, branches0_.tree_id as tree_id3_0_0_ from BagBranch branches0_ where branches0_.tree_id=?

select leaves0_.branch_id as branch_i3_0_1_, leaves0_.id as id1_2_1_, leaves0_.id as id1_2_0_, leaves0_.branch_id as branch_i3_2_0_, leaves0_.index as index2_2_0_ from BagLeaf leaves0_ where leaves0_.branch_id=?
   
select leaves0_.branch_id as branch_i3_0_1_, leaves0_.id as id1_2_1_, leaves0_.id as id1_2_0_, leaves0_.branch_id as branch_i3_2_0_, leaves0_.index as index2_2_0_ from BagLeaf leaves0_ where leaves0_.branch_id=?
   
select branches0_.tree_id as tree_id3_3_1_, branches0_.id as id1_0_1_, branches0_.id as id1_0_0_, branches0_.index as index2_0_0_, branches0_.tree_id as tree_id3_0_0_ from BagBranch branches0_ where branches0_.tree_id=?

select leaves0_.branch_id as branch_i3_0_1_, leaves0_.id as id1_2_1_, leaves0_.id as id1_2_0_, leaves0_.branch_id as branch_i3_2_0_, leaves0_.index as index2_2_0_ from BagLeaf leaves0_ where leaves0_.branch_id=?

select leaves0_.branch_id as branch_i3_0_1_, leaves0_.id as id1_2_1_, leaves0_.id as id1_2_0_, leaves0_.branch_id as branch_i3_2_0_, leaves0_.index as index2_2_0_ from BagLeaf leaves0_ where leaves0_.branch_id=?        

The solution

So, my solution is to simply get the lowest level of children and fetch all needed associations all the way up the entity hierarchy.

When running this JPQL:

List<BagLeaf> leaves = entityManager
.createQuery(
    "select l " +
    "from BagLeaf l " +
    "inner join fetch l.branch b " +
    "inner join fetch b.tree t " +
    "inner join fetch t.forest f " +
    "where f.id = :forestId", BagLeaf.class)
.setParameter("forestId", forestId)
.getResultList();

Hibernate generates only one SQL query:

SELECT bagleaf0_.id        AS id1_2_0_,
       bagbranch1_.id      AS id1_0_1_,
       bagtree2_.id        AS id1_3_2_,
       bagforest3_.id      AS id1_1_3_,
       bagleaf0_.branch_id AS branch_i3_2_0_,
       bagleaf0_.index     AS index2_2_0_,
       bagbranch1_.index   AS index2_0_1_,
       bagbranch1_.tree_id AS tree_id3_0_1_,
       bagtree2_.forest_id AS forest_i3_3_2_,
       bagtree2_.index     AS index2_3_2_
FROM   bagleaf bagleaf0_
INNER JOIN bagbranch bagbranch1_ ON bagleaf0_.branch_id = bagbranch1_.id
INNER JOIN bagtree bagtree2_ ON bagbranch1_.tree_id = bagtree2_.id
INNER JOIN bagforest bagforest3_ ON bagtree2_.forest_id = bagforest3_.id
WHERE  bagforest3_.id = ? 

We get a List of Leaf objects, but each Leaf fetched also the Branch, which fetched the Tree and then the Forest too. Unfortunately, Hibernate can’t magically create the up-down hierarchy from a query result like this.

Trying to access the bags with:

leaves.get(0).getBranch().getTree().getForest().getTrees();

simply throws a LazyInitializationException, since we are trying to access an uninitialized lazy proxy list, outside of an opened Persistence Context.

So, we just need to recreate the Forest hierarchy ourselves from the List of Leaf objects.

And this is how I did it:

EntityGraphBuilder entityGraphBuilder = new EntityGraphBuilder(
    new EntityVisitor[] {
        BagLeaf.ENTITY_VISITOR, 
        BagBranch.ENTITY_VISITOR, 
        BagTree.ENTITY_VISITOR, 
        BagForest.ENTITY_VISITOR
    }
).build(leaves);

ClassId<BagForest> forestClassId = new ClassId<BagForest>(
    BagForest.class, 
    forestId
);

BagForest forest = entityGraphBuilder.getEntityContext().getObject(
    forestClassId
);

The EntityGraphBuilder is one utility I wrote that takes an array of EntityVisitor objects and applies them against the visited objects. This goes recursively up to the Forest object, and we are replacing the Hibernate collections with new ones, adding each child to the parent-children collection.

Since the child collections were replaced, it’s safer not to reattach/merge this object in a new Persistence Context, as all Bags will be marked as dirty.

This is how the Entity uses its visitors:

private <T extends Identifiable, P extends Identifiable> void visit(
        T object) {
    Class<T> clazz = (Class<T>) object.getClass();

    EntityVisitor<T, P> entityVisitor = visitorsMap.get(clazz);

    if (entityVisitor == null) {
        throw new IllegalArgumentException(
            "Class " + clazz + " has no entityVisitor!"
        );
    }

    entityVisitor.visit(object, entityContext);

    P parent = entityVisitor.getParent(object);

    if (parent != null) {
        visit(parent);
    }
} 

And the base EntityVisitor looks like this:

public void visit(T object, EntityContext entityContext) {
    Class<T> clazz = (Class<T>) object.getClass();

    ClassId<T> objectClassId = new ClassId<T>(clazz, object.getId());

    boolean objectVisited = entityContext.isVisited(objectClassId);

    if (!objectVisited) {
        entityContext.visit(objectClassId, object);
    }

    P parent = getParent(object);

    if (parent != null) {
        Class<P> parentClass = (Class<P>) parent.getClass();

        ClassId<P> parentClassId = new ClassId<P>(parentClass, parent.getId());

        if (!entityContext.isVisited(parentClassId)) {
            setChildren(parent);
        }

        List<T> children = getChildren(parent);

        if (!objectVisited) {
            children.add(object);
        }
    }
}

This code is packed as a utility, and the customization comes through extending the EntityVisitors like this:

public static EntityVisitor<BagForest, Identifiable> ENTITY_VISITOR = 
    new EntityVisitor<BagForest, Identifiable>(BagForest.class) {};

public static EntityVisitor<BagTree, BagForest> ENTITY_VISITOR = 
    new EntityVisitor<BagTree, BagForest>(BagTree.class) {
    
    public BagForest getParent(BagTree visitingObject) {
        return visitingObject.getForest();
    }

    public List<BagTree> getChildren(BagForest parent) {
        return parent.getTrees();
    }

    public void setChildren(BagForest parent) {
        parent.setTrees(new ArrayList<BagTree>());
    }
};

public static EntityVisitor<BagBranch, BagTree> ENTITY_VISITOR = 
    new EntityVisitor<BagBranch, BagTree>(BagBranch.class) {
    
    public BagTree getParent(BagBranch visitingObject) {
        return visitingObject.getTree();
    }

    public List<BagBranch> getChildren(BagTree parent) {
        return parent.getBranches();
    }

    public void setChildren(BagTree parent) {
        parent.setBranches(new ArrayList<BagBranch>());
    }
};

public static EntityVisitor<BagLeaf, BagBranch> ENTITY_VISITOR = 
    new EntityVisitor<BagLeaf, BagBranch>(BagLeaf.class) {
    
    public BagBranch getParent(BagLeaf visitingObject) {
        return visitingObject.getBranch();
    }

    public List<BagLeaf> getChildren(BagBranch parent) {
        return parent.getLeaves();
    }

    public void setChildren(BagBranch parent) {
        parent.setLeaves(new ArrayList<BagLeaf>());
    }
};

I'm running an online workshop on the 20-21 and 23-24 of November about High-Performance Java Persistence.

If you enjoyed this article, I bet you are going to love my Book and Video Courses as well.

Conclusion

This is not the Visitor pattern per se, but it has a slight resemblance to it. Although it’s always better to simply use indexed Lists or Sets, you can still get your graph of associations using a single query for Bags too.

Transactions and Concurrency Control eBook

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.