Liquibase Change Types includeAll

Liquibase suggests best practice to organize ChangeLog files is to divide into multiple individual files such as based on features, releases, or other logical parameters. Liquibase provides include Change Type to include multiple segregated ChangeLog files into the main ChangeLog, but if there are multiple ChangeLog files then it becomes difficult to keep track of each file, for example, addition, updation or deleting of ChangeLog file requires updation in main changelog as well.

Liquibase suggests segregating ChageLog files into a directory structure, and provides a includeAll Change Type to include all XML, YAML, and JSON SQL files as individual ChangeLog files of the directory.

includeAll Change Type recursively include all subdirectories of the specified directory in the ChangeLog file.

changelog.mysql.xml

<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xmlns:pro="http://www.liquibase.org/xml/ns/pro" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/pro http://www.liquibase.org/xml/ns/pro/liquibase-pro-latest.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd">
	<includeAll path=".\changelog\tableMetaData\" />
	<includeAll path=".\changelog\tableData\" />
</databaseChangeLog>

.\changelog\tableData\CreateTable.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-1
DROP TABLE IF EXISTS appledb.apple;
CREATE TABLE appledb.apple (apple_id BIGINT AUTO_INCREMENT NOT NULL, apple_name VARCHAR(255) NULL, apple_taste VARCHAR(255) NULL, current_date_time datetime DEFAULT NOW() NULL, CONSTRAINT PK_APPLE PRIMARY KEY (apple_id));

.\changelog\tableData\appleId1.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-3
INSERT INTO `apple` VALUES (2,'Fuji','Salty','2023-09-23 19:06:12');

.\changelog\tableData\appleId2.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-2
INSERT INTO `apple` VALUES (1,'Macintosh','Sweet','2023-09-23 19:06:12');

liquibase.properties (update changeLog File file format with SQL, XML, YAML, JSON as per changelog.mysql.)

changeLogFile=changelog.mysql.xml
liquibase.command.url:jdbc:mysql://localhost:3306/appleDb?createDatabaseIfNotExist=true&autoReconnect=true&useSSL=false&allowPublicKeyRetrieval=true
liquibase.command.username : root
liquibase.command.password : rootroot

Run command liquibase update

The table has been created and records are inserted, the same can be confirmed by running the command.

SELECT * FROM appleDb.apple;

Ignore or Throw an Error if Encounter an Empty Directory

Liqibase Change Type includeAll recursively searches the directory mentioned in the main ChangeLog files for segregated ChangeLog files, but sometimes some invalid or empty directory encounters, Liquibase provides errorIfMissingOrEmpty flag to handle this scenario, by default value is errorIfMissingOrEmpty=false, i.e. throw an error if an empty or invalid directory is encountered, but errorIfMissingOrEmpty=true will ignore the directory and process further execution.

changelog.mysql.xml

<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xmlns:pro="http://www.liquibase.org/xml/ns/pro" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/pro http://www.liquibase.org/xml/ns/pro/liquibase-pro-latest.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd">
	<includeAll path=".\changelog\tableMetaData\" />
	<includeAll path=".\changelog\tableData\" errorIfMissingOrEmpty="false" />
</databaseChangeLog>

.\changelog\tableData\CreateTable.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-1
DROP TABLE IF EXISTS appledb.apple;
CREATE TABLE appledb.apple (apple_id BIGINT AUTO_INCREMENT NOT NULL, apple_name VARCHAR(255) NULL, apple_taste VARCHAR(255) NULL, current_date_time datetime DEFAULT NOW() NULL, CONSTRAINT PK_APPLE PRIMARY KEY (apple_id));

liquibase.properties (update changeLog File file format with SQL, XML, YAML, JSON as per changelog.mysql.)

changeLogFile=changelog.mysql.xml
liquibase.command.url:jdbc:mysql://localhost:3306/appleDb?createDatabaseIfNotExist=true&autoReconnect=true&useSSL=false&allowPublicKeyRetrieval=true
liquibase.command.username : root
liquibase.command.password : rootroot

Run command liquibase update

The table is created but since there is nothing present in \changelog\tableData\ directory, Liquibase still ignores it, the same can be verified by running the command.

SELECT * FROM appleDb.apple;

MinDepth and MaxDepth for Directory Recursion Search

Liquibase provides minDepth and maxDepth flags to control the recursive search of the directory for ChangeLog file in inCludeAll Change Type.

minDept allows controlling the minimum depth of recursion for includeAll, if minDepth=0 which is also the default value, then the search starts from the directory specified in ChangeLog file, if minDepth=1 then the search starts from the first subdirectory.

maxDepth allows to control the maximum depth of recursion for includeAll starting from the base directory, If maxDepth=0 then no subdirectory will be searched, maxDepth can have value up to Integer.MAX_VALUE.

maxDepth should always be greater than minDepth otherwise Liquibase will throw an Error.

changelog.mysql.xml

<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xmlns:pro="http://www.liquibase.org/xml/ns/pro" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd http://www.liquibase.org/xml/ns/pro http://www.liquibase.org/xml/ns/pro/liquibase-pro-latest.xsd http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd">
	<includeAll path=".\changelog\" minDepth="1" maxDepth="1" />
</databaseChangeLog>

.\changelog\CreateTable.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-1
DROP TABLE IF EXISTS appledb.apple;
CREATE TABLE appledb.apple (apple_id BIGINT AUTO_INCREMENT NOT NULL, apple_name VARCHAR(255) NULL, apple_taste VARCHAR(255) NULL, current_date_time datetime DEFAULT NOW() NULL, CONSTRAINT PK_APPLE PRIMARY KEY (apple_id));

.\changelog\appleId1.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-2
INSERT INTO `apple` VALUES (1,'Macintosh','Sweet','2023-09-23 19:06:12');

.\changelog\apple\appleId2.sql

-- liquibase formatted sql

-- changeset shriksha:1695477159989-3
INSERT INTO `apple` VALUES (2,'Fuji','Salty','2023-09-23 19:06:12');

liquibase.properties (update changeLog File file format with SQL, XML, YAML, JSON as per changelog.mysql.)

changeLogFile=changelog.mysql.xml
liquibase.command.url:jdbc:mysql://localhost:3306/appleDb?createDatabaseIfNotExist=true&autoReconnect=true&useSSL=false&allowPublicKeyRetrieval=true
liquibase.command.username : root
liquibase.command.password : rootroot

Run command liquibase update

Liquibase only searches for the current subdirectory which has CreateTable.sql and appleId1.sql file, hence appleId2.sql will be ignored. The same can be verified by running the command.

SELECT * FROM appleDb.apple;