Golang

Upping the Ante: Performing Batch Inserts with GORM

Last time we talked about Batch inserts, we explored a way to achieve that using Go’s SQL standard library[1]. While this approach works well, it’s not the only option available out there. As good engineers, we like to explore our chances and go with the most efficient and reliable solution.

September 6, 2024|

by Tim

Upping the Ante: Performing Batch Inserts with GORM banner

Benchmark Overview

Our application’s repo can be found at https://github.com/wawandco/gorm-batch-insert and it is simply a Go application with two major entities, Gorm Manager(gorman) and GoSQL(gosqlman). Each manager has access to a connection to the database using their corresponding package(gorm - database/sql~ which we’ll refer to as go/sql from now on) and a method which allows them to save contacts in batches.

In addition to those, we also have two benchmark tests, one for each approach, both in the same package. As you know, we’re not allowed to have tests called the same in the same package, hence why we have one of them commented. This is because we’ll be using benchstat, a Go tool to compare benchmark results. Since we’ll be doing this in the form of an A/B comparison then we’ll be using the same name and package to hold both these benchmarks.

Now, let’s take a look at the way we’re setting up the connection for each approach and the way we’re saving those records in batches.

The Results

Once we run both benchmark tests, we can use benchstat to compare both runs, which should lead us to the following results:

The benchstat results consist of several key components, including:

Time/op: This denotes the mean time taken for a specific benchmark operation.
Delta: This indicates the percentage change in performance between two benchmarks.
P-value: A statistical measure that evaluates the likelihood of differences being the result of random chance. N: The number of valid samples considered in the comparison.

When examining the results of benchstat, it’s important to note that a negative percentage in the delta column means that there has been an improvement, while a positive percentage indicates that there has been a decrease in performance. A low p-value implies that the differences are not likely due to chance, indicating that the performance change is statistically significant. To ensure reliability in comparisons, it’s important to consider the sample size (n).

In summary, the GORM way consistently outperforms the go/sql way across various record numbers, with significant speed improvements in most cases. The geomean also indicates a notable performance advantage for GORM over the go/sql approach.

Closing Thoughts

Comparing the first approach which uses the database/sql package and this one it’s kind of like comparing manual assembly to using a ready-made tool. The code employed on the first approach was hard to follow and some constraints as the parameter limit(65.535) aren’t exactly visible during build time.

With GORM, the process of performing batch inserts into a database becomes much simpler and easier to comprehend. Moreover, GORM provides additional flexibility by allowing us to configure the batch size according to our needs. Not to mention, the other features that we’ll be adding to our toolbelt by incorporating GORM in our Go applications.

Remember, sometimes the best solution is not always the one we’re accustomed to, but rather the one that balances performance, readability, and scalability to meet the unique demands of our growing applications.