Git Subtrees

May 20, 2020

Page content

Git subtrees

Simple use of a library

Let us create a Git repository.

~/tmp $ mkdir mainrepo
~/tmp $ cd mainrepo
~/tmp/mainrepo $ git init
Initialized empty Git repository in /home/rat/tmp/mainrepo/.git/

Add some files and commits.

~/tmp/mainrepo $ touch feature1 
~/tmp/mainrepo $ git add *; git commit -m "Add feature1"
~/tmp/mainrepo (master) $ touch feature2
~/tmp/mainrepo (master) $ git add *; git commit -m "Add feature2"

The log shows

~/tmp/mainrepo (master) $ git log --oneline
e2c7cdf Add feature2
f3deaf9 Add feature1

On the side, let us develop a library:

~/tmp/mainrepo (master) $ cd ..
~/tmp/librepo $ mkdir librepo; cd librepo 
~/tmp/librepo $ git init
Initialized empty Git repository in /home/rat/tmp/librepo/.git/
~/tmp/librepo $ touch libfeature1 
~/tmp/librepo $ git add *; git commit -m "Add libfeature1"
~/tmp/librepo $ touch libfeature2 
~/tmp/librepo $ git add *; git commit -m "Add libfeature2"
~/tmp/librepo (master) $ git log --oneline
fb36e6b Add libfeature2
7b2b43e Add libfeature1

This is then published on a bare repository for sharing:

~/tmp $ mkdir lib.git
~/tmp $ cd lib.git
~/tmp/lib.git $ git init --bare 
Initialized empty Git repository in /home/rat/tmp/lib.git/

~/tmp/librepo $ git push /home/rat/tmp/lib.git master
	Counting objects: 9, done.
	Delta compression using up to 4 threads.
	Compressing objects: 100% (7/7), done.
	Writing objects: 100% (9/9), 814 bytes | 0 bytes/s, done.
	Total 9 (delta 3), reused 0 (delta 0)
	To /home/rat/tmp/lib.git
	 * [new branch]      master -> master

Now, in order to use the library in our main project, we try to include it as a subtree. First, to make it more comfortable, we define a “remote” alias.

~/tmp/librepo (master) $ cd ../mainrepo
~/tmp/mainrepo (master) $ git remote add library /home/rat/tmp/librepo/.git
~/tmp/mainrepo (master) $ git remote add library /home/rat/tmp/lib.git   # alternatively

Then add the library:

~/tmp/mainrepo (master) $ git subtree add -P lib library master --squash
git fetch library master
From /home/rat/tmp/librepo/
 * branch            master     -> FETCH_HEAD
 * Added dir 'lib'

The log now shows the creation of a subtree for the library, and merging this to the current master:

	7bddfdd Merge commit '58945f907447e965eadc45c6074bff1a8cce782c' as 'lib'
	58945f9 Squashed 'lib/' content from commit fb36e6b
	e2c7cdf Add feature2
	f3deaf9 Add feature1

Now two things happen. First, development in our repository continues.

~/tmp/mainrepo (master) $ touch feature3
~/tmp/mainrepo (master) $ git add *; git commit -m "Add feature3"
	~/tmp/mainrepo (master) $ echo "changes" >> feature1
~/tmp/mainrepo (master) $ git add *; git commit -m "Update feature1"
~/tmp/mainrepo (master) $ git log --oneline
	b13bf6d Update feature1
	d0a6164 Add feature3
	7bddfdd Merge commit '58945f907447e965eadc45c6074bff1a8cce782c' as 'lib'
	58945f9 Squashed 'lib/' content from commit fb36e6b
	e2c7cdf Add feature2
	f3deaf9 Add feature1

This also happens in the library.

~/tmp/librepo (master) $ touch libfeature3
~/tmp/librepo (master) $ git add *; git commit -m "Add libfeature3"
~/tmp/librepo (master) $ git log --oneline
46b8c30 Add libfeature3
	fb36e6b Add libfeature2
	7b2b43e Add libfeature1

Let us integrate these changes in the library into our project!

~/tmp/mainrepo (master) $ git subtree pull -P lib library master --squash
	remote: Counting objects: 2, done.
	remote: Compressing objects: 100% (2/2), done.
	remote: Total 2 (delta 0), reused 0 (delta 0)
	Unpacking objects: 100% (2/2), done.
	From /home/rat/tmp/librepo/
	 * branch            master     -> FETCH_HEAD
		 fb36e6b..46b8c30  master     -> library/master
	Merge made by the 'recursive' strategy.
	 lib/libfeature3 | 0
	 1 file changed, 0 insertions(+), 0 deletions(-)
	 create mode 100644 lib/libfeature3
~/tmp/mainrepo (master) $ git log --oneline
	88251f1 Merge commit '9b0d431aaec6c321b456e336a625e0ee3fa8d651'
	9b0d431 Squashed 'lib/' changes from fb36e6b..46b8c30
	b13bf6d Update feature1
	d0a6164 Add feature3
	58945f9 Squashed 'lib/' content from commit fb36e6b
	7bddfdd Merge commit '58945f907447e965eadc45c6074bff1a8cce782c' as 'lib'
	e2c7cdf Add feature2
	f3deaf9 Add feature1

This works as expected and we can investigate the changes

~/tmp/mainrepo (master) $ git diff b13bf6d
	diff --git a/lib/libfeature3 b/lib/libfeature3
	new file mode 100644
	index 0000000..e69de29

Advanced use: contributing to a library

As before, development continues in both library and main repository.

~/tmp/librepo (master) $ touch libfeature4
~/tmp/librepo (master) $ git add *; git commit -m "Add libfeature4"
~/tmp/librepo (master) $ git log --oneline
570b0be Add libfeature4
46b8c30 Add libfeature3
	fb36e6b Add libfeature2
	7b2b43e Add libfeature1

~/tmp/mainrepo (master) $ touch feature4
~/tmp/mainrepo (master) $ git add *; git commit -m "Add feature4"
~/tmp/mainrepo (master) $ git log --oneline
	ee67500 Add feature4
	88251f1 Merge commit '9b0d431aaec6c321b456e336a625e0ee3fa8d651'
	9b0d431 Squashed 'lib/' changes from fb36e6b..46b8c30
	b13bf6d Update feature1
	d0a6164 Add feature3
	58945f9 Squashed 'lib/' content from commit fb36e6b
	7bddfdd Merge commit '58945f907447e965eadc45c6074bff1a8cce782c' as 'lib'
	e2c7cdf Add feature2
	f3deaf9 Add feature1

Now we change something inside the library:

~/tmp/mainrepo (master) $ echo "bugfix" >> lib/libfeature2
~/tmp/mainrepo (master) $ git add *; git commit -m "Bugfix in library"

How do we push this to the library? The easiest case is if the library expects a pull-request. We then simply push a new branch which the library owners have to merge (somehow). This is great - it is not our responsibility!

~/tmp/mainrepo (master) $ git subtree push -P lib library branch-for-review --squash
	git push using:  library branch-for-review
	Counting objects: 10, done.
	Delta compression using up to 4 threads.
	Compressing objects: 100% (7/7), done.
	Writing objects: 100% (10/10), 859 bytes | 0 bytes/s, done.
	Total 10 (delta 3), reused 0 (delta 0)
	To /home/rat/tmp/librepo/.git
	 * [new branch]      93510f81b90f7880b3b8cb086574824e2e653f5a -> branch-for-review

If we instead try to push the changes directly to the library, we will fail due to remote changes:

	~/tmp/maincopy (master) $ git subtree push -P lib library master --squash
	git push using:  library master
	To /home/rat/tmp/librepo/.git
	 ! [rejected]        93510f81b90f7880b3b8cb086574824e2e653f5a -> master (fetch first)
	error: failed to push some refs to '/home/rat/tmp/librepo/.git'
	hint: Updates were rejected because the remote contains work that you do
	hint: not have locally. This is usually caused by another repository pushing
	hint: to the same ref. You may want to first integrate the remote changes
	hint: (e.g., 'git pull ...') before pushing again.
	hint: See the 'Note about fast-forwards' in 'git push --help' for details.

So we perform a subtree merge first…

	~/tmp/maincopy (master) $ git subtree pull -P lib library master --squash
	remote: Counting objects: 2, done.
	remote: Compressing objects: 100% (2/2), done.
	remote: Total 2 (delta 1), reused 0 (delta 0)
	Unpacking objects: 100% (2/2), done.
	From /home/rat/tmp/librepo/
	 * branch            master     -> FETCH_HEAD
	 * [new branch]      master     -> library/master
	Merge made by the 'recursive' strategy.
	 lib/libfeature4 | 0
	 1 file changed, 0 insertions(+), 0 deletions(-)
	 create mode 100644 lib/libfeature4

Then we can push this to the (bare) repository, where the bugfix and the merge are added to the history:

	~/tmp/mainrepo (master) $ git subtree push -P lib library2 master --squash
	git push using:  library2 master
	Counting objects: 5, done.
	Delta compression using up to 4 threads.
	Compressing objects: 100% (4/4), done.
	Writing objects: 100% (5/5), 538 bytes | 0 bytes/s, done.
	Total 5 (delta 2), reused 0 (delta 0)
	To /home/rat/tmp/lib.git
		 570b0be..2872ed9  2872ed9313f0224a6a1eddc944a24b9343652e4a -> master

	~/tmp/lib.git (master) $ git log --graph --oneline
	*   2872ed9 Merge commit '37c4398e18eec96a68a5194384edf8be77e47c98'
	|\  
	| * 570b0be Add libfeature4
	* | 93510f8 Bugfix in library
	|/  
	* 46b8c30 Add libfeature3
	* fb36e6b Add libfeature2
	* 7b2b43e Add libfeature1

Can we check beforehand what is send to the remote? Let us try this:

	 ~/tmp/mainrepo (master) $ git subtree split -P lib -b pull-request --squash
	 Created branch 'pull-request'
	 2872ed9313f0224a6a1eddc944a24b9343652e4a

 ~/tmp/mainrepo (master) $ git log --graph pull-request --oneline
	*   2872ed9 Merge commit '37c4398e18eec96a68a5194384edf8be77e47c98'
	|\  
	| * 570b0be Add libfeature4
	* | 93510f8 Bugfix in library
	|/  
	* 46b8c30 Add libfeature3
	* fb36e6b Add libfeature2
	* 7b2b43e Add libfeature1
	rat@m7480 ~/tmp/mainrepo (master) $ git log --graph --oneline
	*   1ed0204 Merge commit '37c4398e18eec96a68a5194384edf8be77e47c98'
	|\  
	| * 37c4398 Squashed 'lib/' changes from 46b8c30..570b0be
	* | aa71d2f Bugfix in library
	* | ee67500 Add feature4
	* |   88251f1 Merge commit '9b0d431aaec6c321b456e336a625e0ee3fa8d651'
	|\ \  
	| |/  
	| * 9b0d431 Squashed 'lib/' changes from fb36e6b..46b8c30
	* | b13bf6d Update feature1
	* | d0a6164 Add feature3
	* |   7bddfdd Merge commit '58945f907447e965eadc45c6074bff1a8cce782c' as 'lib'
	|\ \  
	| |/  
	| * 58945f9 Squashed 'lib/' content from commit fb36e6b
	* e2c7cdf Add feature2
	* f3deaf9 Add feature1

Even more advanced: using branches

This setup has us use a local clone of a bare repository into which we integrate another bare repository as a subtree library, using a development branch….