filter-branch: add --prune-empty to option summary
[git/git.git] / Documentation / git-filter-branch.txt
CommitLineData
c401b33c
JS
1git-filter-branch(1)
2====================
3
4NAME
5----
6git-filter-branch - Rewrite branches
7
8SYNOPSIS
9--------
10[verse]
b1889c36 11'git filter-branch' [--env-filter <command>] [--tree-filter <command>]
c401b33c
JS
12 [--index-filter <command>] [--parent-filter <command>]
13 [--msg-filter <command>] [--commit-filter <command>]
14 [--tag-name-filter <command>] [--subdirectory-filter <directory>]
e3679ab4 15 [--prune-empty]
5433235d 16 [--original <namespace>] [-d <directory>] [-f | --force]
8afa4210 17 [--] [<rev-list options>...]
c401b33c
JS
18
19DESCRIPTION
20-----------
08203668
JS
21Lets you rewrite git revision history by rewriting the branches mentioned
22in the <rev-list options>, applying custom filters on each revision.
c401b33c
JS
23Those filters can modify each tree (e.g. removing a file or running
24a perl rewrite on all files) or information about each commit.
25Otherwise, all information (including original commit times or merge
26information) will be preserved.
27
08203668 28The command will only rewrite the _positive_ refs mentioned in the
bf7c9021 29command line (e.g. if you pass 'a..b', only 'b' will be rewritten).
08203668
JS
30If you specify no filters, the commits will be recommitted without any
31changes, which would normally have no effect. Nevertheless, this may be
32useful in the future for compensating for some git bugs or such,
33therefore such a usage is permitted.
c401b33c 34
c6d8f763
DCS
35*NOTE*: This command honors `.git/info/grafts`. If you have any grafts
36defined, running this command will make them permanent.
37
73616fd3 38*WARNING*! The rewritten history will have different object names for all
c401b33c
JS
39the objects and will not converge with the original branch. You will not
40be able to easily push and distribute the rewritten branch on top of the
41original branch. Please do not use this command if you do not know the
42full implications, and avoid using it anyway, if a simple single commit
97c33c65
TR
43would suffice to fix your problem. (See the "RECOVERING FROM UPSTREAM
44REBASE" section in linkgit:git-rebase[1] for further information about
45rewriting published history.)
c401b33c 46
dfd05e38
JS
47Always verify that the rewritten version is correct: The original refs,
48if different from the rewritten ones, will be stored in the namespace
49'refs/original/'.
c401b33c 50
bf7c9021 51Note that since this operation is very I/O expensive, it might
08203668
JS
52be a good idea to redirect the temporary directory off-disk with the
53'-d' option, e.g. on tmpfs. Reportedly the speedup is very noticeable.
c401b33c
JS
54
55
56Filters
57~~~~~~~
58
59The filters are applied in the order as listed below. The <command>
bf7c9021
RW
60argument is always evaluated in the shell context using the 'eval' command
61(with the notable exception of the commit filter, for technical reasons).
c401b33c
JS
62Prior to that, the $GIT_COMMIT environment variable will be set to contain
63the id of the commit being rewritten. Also, GIT_AUTHOR_NAME,
64GIT_AUTHOR_EMAIL, GIT_AUTHOR_DATE, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL,
bf7c9021
RW
65and GIT_COMMITTER_DATE are set according to the current commit. The values
66of these variables after the filters have run, are used for the new commit.
67If any evaluation of <command> returns a non-zero exit status, the whole
68operation will be aborted.
c401b33c
JS
69
70A 'map' function is available that takes an "original sha1 id" argument
71and outputs a "rewritten sha1 id" if the commit has been already
32c37c12
JS
72rewritten, and "original sha1 id" otherwise; the 'map' function can
73return several ids on separate lines if your commit filter emitted
74multiple commits.
c401b33c
JS
75
76
77OPTIONS
78-------
79
80--env-filter <command>::
bf7c9021
RW
81 This filter may be used if you only need to modify the environment
82 in which the commit will be performed. Specifically, you might
83 want to rewrite the author/committer name/email/time environment
5162e697 84 variables (see linkgit:git-commit[1] for details). Do not forget
c401b33c
JS
85 to re-export the variables.
86
87--tree-filter <command>::
88 This is the filter for rewriting the tree and its contents.
89 The argument is evaluated in shell with the working
90 directory set to the root of the checked out tree. The new tree
91 is then used as-is (new files are auto-added, disappeared files
92 are auto-removed - neither .gitignore files nor any other ignore
73616fd3 93 rules *HAVE ANY EFFECT*!).
c401b33c
JS
94
95--index-filter <command>::
96 This is the filter for rewriting the index. It is similar to the
97 tree filter but does not check out the tree, which makes it much
3bc427e0
TR
98 faster. Frequently used with `git rm \--cached
99 \--ignore-unmatch ...`, see EXAMPLES below. For hairy
100 cases, see linkgit:git-update-index[1].
c401b33c
JS
101
102--parent-filter <command>::
103 This is the filter for rewriting the commit's parent list.
104 It will receive the parent string on stdin and shall output
105 the new parent string on stdout. The parent string is in
483bc4f0 106 the format described in linkgit:git-commit-tree[1]: empty for
c401b33c
JS
107 the initial commit, "-p parent" for a normal commit and
108 "-p parent1 -p parent2 -p parent3 ..." for a merge commit.
109
110--msg-filter <command>::
111 This is the filter for rewriting the commit messages.
112 The argument is evaluated in the shell with the original
113 commit message on standard input; its standard output is
114 used as the new commit message.
115
116--commit-filter <command>::
117 This is the filter for performing the commit.
118 If this filter is specified, it will be called instead of the
ba020ef5 119 'git-commit-tree' command, with arguments of the form
c401b33c
JS
120 "<TREE_ID> [-p <PARENT_COMMIT_ID>]..." and the log message on
121 stdin. The commit id is expected on stdout.
122+
123As a special extension, the commit filter may emit multiple
c5833f6e 124commit ids; in that case, the rewritten children of the original commit will
c401b33c 125have all of them as parents.
f95eef15
JS
126+
127You can use the 'map' convenience function in this filter, and other
128convenience functions, too. For example, calling 'skip_commit "$@"'
129will leave out the current commit (but not its changes! If you want
ba020ef5 130that, use 'git-rebase' instead).
d3240d93
PH
131+
132You can also use the 'git_commit_non_empty_tree "$@"' instead of
133'git commit-tree "$@"' if you don't wish to keep commits with a single parent
134and that makes no change to the tree.
c401b33c
JS
135
136--tag-name-filter <command>::
137 This is the filter for rewriting tag names. When passed,
138 it will be called for every tag ref that points to a rewritten
139 object (or to a tag object which points to a rewritten object).
140 The original tag name is passed via standard input, and the new
141 tag name is expected on standard output.
142+
143The original tags are not deleted, but can be overwritten;
5876b8ee 144use "--tag-name-filter cat" to simply update the tags. In this
c401b33c
JS
145case, be very careful and make sure you have the old tags
146backed up in case the conversion has run afoul.
147+
1bf6551e
BC
148Nearly proper rewriting of tag objects is supported. If the tag has
149a message attached, a new tag object will be created with the same message,
150author, and timestamp. If the tag has a signature attached, the
151signature will be stripped. It is by definition impossible to preserve
152signatures. The reason this is "nearly" proper, is because ideally if
153the tag did not change (points to the same object, has the same name, etc.)
154it should retain any signature. That is not the case, signatures will always
155be removed, buyer beware. There is also no support for changing the
156author or timestamp (or the tag message for that matter). Tags which point
157to other tags will be rewritten to point to the underlying commit.
c401b33c
JS
158
159--subdirectory-filter <directory>::
73616fd3
JS
160 Only look at the history which touches the given subdirectory.
161 The result will contain that directory (and only that) as its
162 project root.
c401b33c 163
d3240d93
PH
164--prune-empty::
165 Some kind of filters will generate empty commits, that left the tree
166 untouched. This switch allow git-filter-branch to ignore such
167 commits. Though, this switch only applies for commits that have one
168 and only one parent, it will hence keep merges points. Also, this
169 option is not compatible with the use of '--commit-filter'. Though you
170 just need to use the function 'git_commit_non_empty_tree "$@"' instead
171 of the 'git commit-tree "$@"' idiom in your commit filter to make that
172 happen.
173
5433235d
GB
174--original <namespace>::
175 Use this option to set the namespace where the original commits
176 will be stored. The default value is 'refs/original'.
177
c401b33c
JS
178-d <directory>::
179 Use this option to set the path to the temporary directory used for
180 rewriting. When applying a tree filter, the command needs to
bf7c9021 181 temporarily check out the tree to some directory, which may consume
c401b33c
JS
182 considerable space in case of large projects. By default it
183 does this in the '.git-rewrite/' directory but you can override
184 that choice by this parameter.
185
3240240f
SB
186-f::
187--force::
ba020ef5 188 'git-filter-branch' refuses to start with an existing temporary
dfd05e38
JS
189 directory or when there are already refs starting with
190 'refs/original/', unless forced.
191
f448e24e 192<rev-list options>...::
8afa4210
TR
193 Arguments for 'git-rev-list'. All positive refs included by
194 these options are rewritten. You may also specify options
195 such as '--all', but you must use '--' to separate them from
196 the 'git-filter-branch' options.
c401b33c
JS
197
198
199Examples
200--------
201
202Suppose you want to remove a file (containing confidential information
203or copyright violation) from all commits:
204
205-------------------------------------------------------
dfd05e38 206git filter-branch --tree-filter 'rm filename' HEAD
c401b33c
JS
207-------------------------------------------------------
208
e4d594c6
JL
209However, if the file is absent from the tree of some commit,
210a simple `rm filename` will fail for that tree and commit.
211Thus you may instead want to use `rm -f filename` as the script.
212
3bc427e0
TR
213Using `\--index-filter` with 'git-rm' yields a significantly faster
214version. Like with using `rm filename`, `git rm --cached filename`
215will fail if the file is absent from the tree of a commit. If you
216want to "completely forget" a file, it does not matter when it entered
217history, so we also add `\--ignore-unmatch`:
c401b33c 218
dfd05e38 219--------------------------------------------------------------------------
3bc427e0 220git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD
dfd05e38 221--------------------------------------------------------------------------
c401b33c 222
8ef44519 223Now, you will get the rewritten history saved in HEAD.
c401b33c 224
8afa4210
TR
225To rewrite the repository to look as if `foodir/` had been its project
226root, and discard all other history:
227
228-------------------------------------------------------
229git filter-branch --subdirectory-filter foodir -- --all
230-------------------------------------------------------
231
232Thus you can, e.g., turn a library subdirectory into a repository of
233its own. Note the `\--` that separates 'filter-branch' options from
234revision options, and the `\--all` to rewrite all branches and tags.
235
32c37c12
JS
236To set a commit (which typically is at the tip of another
237history) to be the parent of the current initial commit, in
238order to paste the other history behind the current history:
c401b33c 239
dfd05e38
JS
240-------------------------------------------------------------------
241git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD
242-------------------------------------------------------------------
c401b33c 243
08203668
JS
244(if the parent string is empty - which happens when we are dealing with
245the initial commit - add graftcommit as a parent). Note that this assumes
c401b33c
JS
246history with a single root (that is, no merge without common ancestors
247happened). If this is not the case, use:
248
dfd05e38 249--------------------------------------------------------------------------
c401b33c 250git filter-branch --parent-filter \
41e86a37 251 'test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>" || cat' HEAD
dfd05e38 252--------------------------------------------------------------------------
c401b33c 253
32c37c12
JS
254or even simpler:
255
256-----------------------------------------------
257echo "$commit-id $graft-id" >> .git/info/grafts
dfd05e38 258git filter-branch $graft-id..HEAD
32c37c12
JS
259-----------------------------------------------
260
c401b33c
JS
261To remove commits authored by "Darl McBribe" from the history:
262
263------------------------------------------------------------------------------
264git filter-branch --commit-filter '
265 if [ "$GIT_AUTHOR_NAME" = "Darl McBribe" ];
266 then
f95eef15 267 skip_commit "$@";
c401b33c
JS
268 else
269 git commit-tree "$@";
dfd05e38 270 fi' HEAD
c401b33c
JS
271------------------------------------------------------------------------------
272
8451c565 273The function 'skip_commit' is defined as follows:
f95eef15
JS
274
275--------------------------
276skip_commit()
277{
278 shift;
279 while [ -n "$1" ];
280 do
281 shift;
282 map "$1";
283 shift;
284 done;
285}
286--------------------------
287
c401b33c
JS
288The shift magic first throws away the tree id and then the -p
289parameters. Note that this handles merges properly! In case Darl
290committed a merge between P1 and P2, it will be propagated properly
291and all children of the merge will become merge commits with P1,P2
292as their parents instead of the merge commit.
293
a1748890 294You can rewrite the commit log messages using `--msg-filter`. For
ba020ef5 295example, 'git-svn-id' strings in a repository created by 'git-svn' can
ed10d9aa
MV
296be removed this way:
297
298-------------------------------------------------------
a1748890 299git filter-branch --msg-filter '
ed10d9aa
MV
300 sed -e "/^git-svn-id:/d"
301'
302-------------------------------------------------------
f95eef15 303
c401b33c
JS
304To restrict rewriting to only part of the history, specify a revision
305range in addition to the new branch name. The new branch name will
ba020ef5 306point to the top-most revision that a 'git-rev-list' of this range
c401b33c
JS
307will print.
308
b8f42332
JS
309If you need to add 'Acked-by' lines to, say, the last 10 commits (none
310of which is a merge), use this command:
311
312--------------------------------------------------------
313git filter-branch --msg-filter '
314 cat &&
315 echo "Acked-by: Bugs Bunny <bunny@bugzilla.org>"
316' HEAD~10..HEAD
317--------------------------------------------------------
318
08203668
JS
319*NOTE* the changes introduced by the commits, and which are not reverted
320by subsequent commits, will still be in the rewritten branch. If you want
c401b33c 321to throw out _changes_ together with the commits, you should use the
ba020ef5 322interactive mode of 'git-rebase'.
c401b33c 323
08203668 324
c401b33c
JS
325Consider this history:
326
327------------------
328 D--E--F--G--H
329 / /
330A--B-----C
331------------------
332
333To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use:
334
335--------------------------------
dfd05e38 336git filter-branch ... C..H
c401b33c
JS
337--------------------------------
338
339To rewrite commits E,F,G,H, use one of these:
340
341----------------------------------------
dfd05e38
JS
342git filter-branch ... C..H --not D
343git filter-branch ... D..H --not C
c401b33c
JS
344----------------------------------------
345
346To move the whole tree into a subdirectory, or remove it from there:
347
348---------------------------------------------------------------
349git filter-branch --index-filter \
350 'git ls-files -s | sed "s-\t-&newsubdir/-" |
351 GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
352 git update-index --index-info &&
dfd05e38 353 mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD
c401b33c
JS
354---------------------------------------------------------------
355
356
d0268de6
TR
357
358Checklist for Shrinking a Repository
359------------------------------------
360
361git-filter-branch is often used to get rid of a subset of files,
362usually with some combination of `\--index-filter` and
363`\--subdirectory-filter`. People expect the resulting repository to
364be smaller than the original, but you need a few more steps to
365actually make it smaller, because git tries hard not to lose your
366objects until you tell it to. First make sure that:
367
368* You really removed all variants of a filename, if a blob was moved
369 over its lifetime. `git log \--name-only \--follow \--all \--
370 filename` can help you find renames.
371
372* You really filtered all refs: use `\--tag-name-filter cat \--
373 \--all` when calling git-filter-branch.
374
375Then there are two ways to get a smaller repository. A safer way is
376to clone, that keeps your original intact.
377
378* Clone it with `git clone +++file:///path/to/repo+++`. The clone
379 will not have the removed objects. See linkgit:git-clone[1]. (Note
380 that cloning with a plain path just hardlinks everything!)
381
382If you really don't want to clone it, for whatever reasons, check the
383following points instead (in this order). This is a very destructive
384approach, so *make a backup* or go back to cloning it. You have been
385warned.
386
387* Remove the original refs backed up by git-filter-branch: say `git
388 for-each-ref \--format="%(refname)" refs/original/ | xargs -n 1 git
389 update-ref -d`.
390
391* Expire all reflogs with `git reflog expire \--expire=now \--all`.
392
393* Garbage collect all unreferenced objects with `git gc \--prune=now`
394 (or if your git-gc is not new enough to support arguments to
395 `\--prune`, use `git repack -ad; git prune` instead).
396
397
c401b33c
JS
398Author
399------
400Written by Petr "Pasky" Baudis <pasky@suse.cz>,
401and the git list <git@vger.kernel.org>
402
403Documentation
404--------------
405Documentation by Petr Baudis and the git list.
406
407GIT
408---
9e1f0a85 409Part of the linkgit:git[1] suite