filter-branch: add --prune-empty to option summary
[git/git.git] / Documentation / git-filter-branch.txt
1 git-filter-branch(1)
2 ====================
3
4 NAME
5 ----
6 git-filter-branch - Rewrite branches
7
8 SYNOPSIS
9 --------
10 [verse]
11 'git filter-branch' [--env-filter <command>] [--tree-filter <command>]
12 [--index-filter <command>] [--parent-filter <command>]
13 [--msg-filter <command>] [--commit-filter <command>]
14 [--tag-name-filter <command>] [--subdirectory-filter <directory>]
15 [--prune-empty]
16 [--original <namespace>] [-d <directory>] [-f | --force]
17 [--] [<rev-list options>...]
18
19 DESCRIPTION
20 -----------
21 Lets you rewrite git revision history by rewriting the branches mentioned
22 in the <rev-list options>, applying custom filters on each revision.
23 Those filters can modify each tree (e.g. removing a file or running
24 a perl rewrite on all files) or information about each commit.
25 Otherwise, all information (including original commit times or merge
26 information) will be preserved.
27
28 The command will only rewrite the _positive_ refs mentioned in the
29 command line (e.g. if you pass 'a..b', only 'b' will be rewritten).
30 If you specify no filters, the commits will be recommitted without any
31 changes, which would normally have no effect. Nevertheless, this may be
32 useful in the future for compensating for some git bugs or such,
33 therefore such a usage is permitted.
34
35 *NOTE*: This command honors `.git/info/grafts`. If you have any grafts
36 defined, running this command will make them permanent.
37
38 *WARNING*! The rewritten history will have different object names for all
39 the objects and will not converge with the original branch. You will not
40 be able to easily push and distribute the rewritten branch on top of the
41 original branch. Please do not use this command if you do not know the
42 full implications, and avoid using it anyway, if a simple single commit
43 would suffice to fix your problem. (See the "RECOVERING FROM UPSTREAM
44 REBASE" section in linkgit:git-rebase[1] for further information about
45 rewriting published history.)
46
47 Always verify that the rewritten version is correct: The original refs,
48 if different from the rewritten ones, will be stored in the namespace
49 'refs/original/'.
50
51 Note that since this operation is very I/O expensive, it might
52 be a good idea to redirect the temporary directory off-disk with the
53 '-d' option, e.g. on tmpfs. Reportedly the speedup is very noticeable.
54
55
56 Filters
57 ~~~~~~~
58
59 The filters are applied in the order as listed below. The <command>
60 argument is always evaluated in the shell context using the 'eval' command
61 (with the notable exception of the commit filter, for technical reasons).
62 Prior to that, the $GIT_COMMIT environment variable will be set to contain
63 the id of the commit being rewritten. Also, GIT_AUTHOR_NAME,
64 GIT_AUTHOR_EMAIL, GIT_AUTHOR_DATE, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL,
65 and GIT_COMMITTER_DATE are set according to the current commit. The values
66 of these variables after the filters have run, are used for the new commit.
67 If any evaluation of <command> returns a non-zero exit status, the whole
68 operation will be aborted.
69
70 A 'map' function is available that takes an "original sha1 id" argument
71 and outputs a "rewritten sha1 id" if the commit has been already
72 rewritten, and "original sha1 id" otherwise; the 'map' function can
73 return several ids on separate lines if your commit filter emitted
74 multiple commits.
75
76
77 OPTIONS
78 -------
79
80 --env-filter <command>::
81 This filter may be used if you only need to modify the environment
82 in which the commit will be performed. Specifically, you might
83 want to rewrite the author/committer name/email/time environment
84 variables (see linkgit:git-commit[1] for details). Do not forget
85 to re-export the variables.
86
87 --tree-filter <command>::
88 This is the filter for rewriting the tree and its contents.
89 The argument is evaluated in shell with the working
90 directory set to the root of the checked out tree. The new tree
91 is then used as-is (new files are auto-added, disappeared files
92 are auto-removed - neither .gitignore files nor any other ignore
93 rules *HAVE ANY EFFECT*!).
94
95 --index-filter <command>::
96 This is the filter for rewriting the index. It is similar to the
97 tree filter but does not check out the tree, which makes it much
98 faster. Frequently used with `git rm \--cached
99 \--ignore-unmatch ...`, see EXAMPLES below. For hairy
100 cases, see linkgit:git-update-index[1].
101
102 --parent-filter <command>::
103 This is the filter for rewriting the commit's parent list.
104 It will receive the parent string on stdin and shall output
105 the new parent string on stdout. The parent string is in
106 the format described in linkgit:git-commit-tree[1]: empty for
107 the initial commit, "-p parent" for a normal commit and
108 "-p parent1 -p parent2 -p parent3 ..." for a merge commit.
109
110 --msg-filter <command>::
111 This is the filter for rewriting the commit messages.
112 The argument is evaluated in the shell with the original
113 commit message on standard input; its standard output is
114 used as the new commit message.
115
116 --commit-filter <command>::
117 This is the filter for performing the commit.
118 If this filter is specified, it will be called instead of the
119 'git-commit-tree' command, with arguments of the form
120 "<TREE_ID> [-p <PARENT_COMMIT_ID>]..." and the log message on
121 stdin. The commit id is expected on stdout.
122 +
123 As a special extension, the commit filter may emit multiple
124 commit ids; in that case, the rewritten children of the original commit will
125 have all of them as parents.
126 +
127 You can use the 'map' convenience function in this filter, and other
128 convenience functions, too. For example, calling 'skip_commit "$@"'
129 will leave out the current commit (but not its changes! If you want
130 that, use 'git-rebase' instead).
131 +
132 You can also use the 'git_commit_non_empty_tree "$@"' instead of
133 'git commit-tree "$@"' if you don't wish to keep commits with a single parent
134 and that makes no change to the tree.
135
136 --tag-name-filter <command>::
137 This is the filter for rewriting tag names. When passed,
138 it will be called for every tag ref that points to a rewritten
139 object (or to a tag object which points to a rewritten object).
140 The original tag name is passed via standard input, and the new
141 tag name is expected on standard output.
142 +
143 The original tags are not deleted, but can be overwritten;
144 use "--tag-name-filter cat" to simply update the tags. In this
145 case, be very careful and make sure you have the old tags
146 backed up in case the conversion has run afoul.
147 +
148 Nearly proper rewriting of tag objects is supported. If the tag has
149 a message attached, a new tag object will be created with the same message,
150 author, and timestamp. If the tag has a signature attached, the
151 signature will be stripped. It is by definition impossible to preserve
152 signatures. The reason this is "nearly" proper, is because ideally if
153 the tag did not change (points to the same object, has the same name, etc.)
154 it should retain any signature. That is not the case, signatures will always
155 be removed, buyer beware. There is also no support for changing the
156 author or timestamp (or the tag message for that matter). Tags which point
157 to other tags will be rewritten to point to the underlying commit.
158
159 --subdirectory-filter <directory>::
160 Only look at the history which touches the given subdirectory.
161 The result will contain that directory (and only that) as its
162 project root.
163
164 --prune-empty::
165 Some kind of filters will generate empty commits, that left the tree
166 untouched. This switch allow git-filter-branch to ignore such
167 commits. Though, this switch only applies for commits that have one
168 and only one parent, it will hence keep merges points. Also, this
169 option is not compatible with the use of '--commit-filter'. Though you
170 just need to use the function 'git_commit_non_empty_tree "$@"' instead
171 of the 'git commit-tree "$@"' idiom in your commit filter to make that
172 happen.
173
174 --original <namespace>::
175 Use this option to set the namespace where the original commits
176 will be stored. The default value is 'refs/original'.
177
178 -d <directory>::
179 Use this option to set the path to the temporary directory used for
180 rewriting. When applying a tree filter, the command needs to
181 temporarily check out the tree to some directory, which may consume
182 considerable space in case of large projects. By default it
183 does this in the '.git-rewrite/' directory but you can override
184 that choice by this parameter.
185
186 -f::
187 --force::
188 'git-filter-branch' refuses to start with an existing temporary
189 directory or when there are already refs starting with
190 'refs/original/', unless forced.
191
192 <rev-list options>...::
193 Arguments for 'git-rev-list'. All positive refs included by
194 these options are rewritten. You may also specify options
195 such as '--all', but you must use '--' to separate them from
196 the 'git-filter-branch' options.
197
198
199 Examples
200 --------
201
202 Suppose you want to remove a file (containing confidential information
203 or copyright violation) from all commits:
204
205 -------------------------------------------------------
206 git filter-branch --tree-filter 'rm filename' HEAD
207 -------------------------------------------------------
208
209 However, if the file is absent from the tree of some commit,
210 a simple `rm filename` will fail for that tree and commit.
211 Thus you may instead want to use `rm -f filename` as the script.
212
213 Using `\--index-filter` with 'git-rm' yields a significantly faster
214 version. Like with using `rm filename`, `git rm --cached filename`
215 will fail if the file is absent from the tree of a commit. If you
216 want to "completely forget" a file, it does not matter when it entered
217 history, so we also add `\--ignore-unmatch`:
218
219 --------------------------------------------------------------------------
220 git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD
221 --------------------------------------------------------------------------
222
223 Now, you will get the rewritten history saved in HEAD.
224
225 To rewrite the repository to look as if `foodir/` had been its project
226 root, and discard all other history:
227
228 -------------------------------------------------------
229 git filter-branch --subdirectory-filter foodir -- --all
230 -------------------------------------------------------
231
232 Thus you can, e.g., turn a library subdirectory into a repository of
233 its own. Note the `\--` that separates 'filter-branch' options from
234 revision options, and the `\--all` to rewrite all branches and tags.
235
236 To set a commit (which typically is at the tip of another
237 history) to be the parent of the current initial commit, in
238 order to paste the other history behind the current history:
239
240 -------------------------------------------------------------------
241 git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD
242 -------------------------------------------------------------------
243
244 (if the parent string is empty - which happens when we are dealing with
245 the initial commit - add graftcommit as a parent). Note that this assumes
246 history with a single root (that is, no merge without common ancestors
247 happened). If this is not the case, use:
248
249 --------------------------------------------------------------------------
250 git filter-branch --parent-filter \
251 'test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>" || cat' HEAD
252 --------------------------------------------------------------------------
253
254 or even simpler:
255
256 -----------------------------------------------
257 echo "$commit-id $graft-id" >> .git/info/grafts
258 git filter-branch $graft-id..HEAD
259 -----------------------------------------------
260
261 To remove commits authored by "Darl McBribe" from the history:
262
263 ------------------------------------------------------------------------------
264 git filter-branch --commit-filter '
265 if [ "$GIT_AUTHOR_NAME" = "Darl McBribe" ];
266 then
267 skip_commit "$@";
268 else
269 git commit-tree "$@";
270 fi' HEAD
271 ------------------------------------------------------------------------------
272
273 The function 'skip_commit' is defined as follows:
274
275 --------------------------
276 skip_commit()
277 {
278 shift;
279 while [ -n "$1" ];
280 do
281 shift;
282 map "$1";
283 shift;
284 done;
285 }
286 --------------------------
287
288 The shift magic first throws away the tree id and then the -p
289 parameters. Note that this handles merges properly! In case Darl
290 committed a merge between P1 and P2, it will be propagated properly
291 and all children of the merge will become merge commits with P1,P2
292 as their parents instead of the merge commit.
293
294 You can rewrite the commit log messages using `--msg-filter`. For
295 example, 'git-svn-id' strings in a repository created by 'git-svn' can
296 be removed this way:
297
298 -------------------------------------------------------
299 git filter-branch --msg-filter '
300 sed -e "/^git-svn-id:/d"
301 '
302 -------------------------------------------------------
303
304 To restrict rewriting to only part of the history, specify a revision
305 range in addition to the new branch name. The new branch name will
306 point to the top-most revision that a 'git-rev-list' of this range
307 will print.
308
309 If you need to add 'Acked-by' lines to, say, the last 10 commits (none
310 of which is a merge), use this command:
311
312 --------------------------------------------------------
313 git filter-branch --msg-filter '
314 cat &&
315 echo "Acked-by: Bugs Bunny <bunny@bugzilla.org>"
316 ' HEAD~10..HEAD
317 --------------------------------------------------------
318
319 *NOTE* the changes introduced by the commits, and which are not reverted
320 by subsequent commits, will still be in the rewritten branch. If you want
321 to throw out _changes_ together with the commits, you should use the
322 interactive mode of 'git-rebase'.
323
324
325 Consider this history:
326
327 ------------------
328 D--E--F--G--H
329 / /
330 A--B-----C
331 ------------------
332
333 To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use:
334
335 --------------------------------
336 git filter-branch ... C..H
337 --------------------------------
338
339 To rewrite commits E,F,G,H, use one of these:
340
341 ----------------------------------------
342 git filter-branch ... C..H --not D
343 git filter-branch ... D..H --not C
344 ----------------------------------------
345
346 To move the whole tree into a subdirectory, or remove it from there:
347
348 ---------------------------------------------------------------
349 git filter-branch --index-filter \
350 'git ls-files -s | sed "s-\t-&newsubdir/-" |
351 GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
352 git update-index --index-info &&
353 mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD
354 ---------------------------------------------------------------
355
356
357
358 Checklist for Shrinking a Repository
359 ------------------------------------
360
361 git-filter-branch is often used to get rid of a subset of files,
362 usually with some combination of `\--index-filter` and
363 `\--subdirectory-filter`. People expect the resulting repository to
364 be smaller than the original, but you need a few more steps to
365 actually make it smaller, because git tries hard not to lose your
366 objects until you tell it to. First make sure that:
367
368 * You really removed all variants of a filename, if a blob was moved
369 over its lifetime. `git log \--name-only \--follow \--all \--
370 filename` can help you find renames.
371
372 * You really filtered all refs: use `\--tag-name-filter cat \--
373 \--all` when calling git-filter-branch.
374
375 Then there are two ways to get a smaller repository. A safer way is
376 to clone, that keeps your original intact.
377
378 * Clone it with `git clone +++file:///path/to/repo+++`. The clone
379 will not have the removed objects. See linkgit:git-clone[1]. (Note
380 that cloning with a plain path just hardlinks everything!)
381
382 If you really don't want to clone it, for whatever reasons, check the
383 following points instead (in this order). This is a very destructive
384 approach, so *make a backup* or go back to cloning it. You have been
385 warned.
386
387 * Remove the original refs backed up by git-filter-branch: say `git
388 for-each-ref \--format="%(refname)" refs/original/ | xargs -n 1 git
389 update-ref -d`.
390
391 * Expire all reflogs with `git reflog expire \--expire=now \--all`.
392
393 * Garbage collect all unreferenced objects with `git gc \--prune=now`
394 (or if your git-gc is not new enough to support arguments to
395 `\--prune`, use `git repack -ad; git prune` instead).
396
397
398 Author
399 ------
400 Written by Petr "Pasky" Baudis <pasky@suse.cz>,
401 and the git list <git@vger.kernel.org>
402
403 Documentation
404 --------------
405 Documentation by Petr Baudis and the git list.
406
407 GIT
408 ---
409 Part of the linkgit:git[1] suite