serve: introduce the server-option capability
[git/git.git] / Documentation / technical / protocol-v2.txt
CommitLineData
ed10cb95
BW
1 Git Wire Protocol, Version 2
2==============================
3
4This document presents a specification for a version 2 of Git's wire
5protocol. Protocol v2 will improve upon v1 in the following ways:
6
7 * Instead of multiple service names, multiple commands will be
8 supported by a single service
9 * Easily extendable as capabilities are moved into their own section
10 of the protocol, no longer being hidden behind a NUL byte and
11 limited by the size of a pkt-line
12 * Separate out other information hidden behind NUL bytes (e.g. agent
13 string as a capability and symrefs can be requested using 'ls-refs')
14 * Reference advertisement will be omitted unless explicitly requested
15 * ls-refs command to explicitly request some refs
16 * Designed with http and stateless-rpc in mind. With clear flush
17 semantics the http remote helper can simply act as a proxy
18
19In protocol v2 communication is command oriented. When first contacting a
20server a list of capabilities will advertised. Some of these capabilities
21will be commands which a client can request be executed. Once a command
22has completed, a client can reuse the connection and request that other
23commands be executed.
24
25 Packet-Line Framing
26---------------------
27
28All communication is done using packet-line framing, just as in v1. See
29`Documentation/technical/pack-protocol.txt` and
30`Documentation/technical/protocol-common.txt` for more information.
31
32In protocol v2 these special packets will have the following semantics:
33
34 * '0000' Flush Packet (flush-pkt) - indicates the end of a message
35 * '0001' Delimiter Packet (delim-pkt) - separates sections of a message
36
37 Initial Client Request
38------------------------
39
40In general a client can request to speak protocol v2 by sending
41`version=2` through the respective side-channel for the transport being
42used which inevitably sets `GIT_PROTOCOL`. More information can be
43found in `pack-protocol.txt` and `http-protocol.txt`. In all cases the
44response from the server is the capability advertisement.
45
46 Git Transport
47~~~~~~~~~~~~~~~
48
49When using the git:// transport, you can request to use protocol v2 by
50sending "version=2" as an extra parameter:
51
52 003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
53
54 SSH and File Transport
55~~~~~~~~~~~~~~~~~~~~~~~~
56
57When using either the ssh:// or file:// transport, the GIT_PROTOCOL
58environment variable must be set explicitly to include "version=2".
59
60 HTTP Transport
61~~~~~~~~~~~~~~~~
62
63When using the http:// or https:// transport a client makes a "smart"
64info/refs request as described in `http-protocol.txt` and requests that
65v2 be used by supplying "version=2" in the `Git-Protocol` header.
66
67 C: Git-Protocol: version=2
68 C:
69 C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
70
71A v2 server would reply:
72
73 S: 200 OK
74 S: <Some headers>
75 S: ...
76 S:
77 S: 000eversion 2\n
78 S: <capability-advertisement>
79
80Subsequent requests are then made directly to the service
81`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
82
83 Capability Advertisement
84--------------------------
85
86A server which decides to communicate (based on a request from a client)
87using protocol version 2, notifies the client by sending a version string
88in its initial response followed by an advertisement of its capabilities.
89Each capability is a key with an optional value. Clients must ignore all
90unknown keys. Semantics of unknown values are left to the definition of
91each key. Some capabilities will describe commands which can be requested
92to be executed by the client.
93
94 capability-advertisement = protocol-version
95 capability-list
96 flush-pkt
97
98 protocol-version = PKT-LINE("version 2" LF)
99 capability-list = *capability
100 capability = PKT-LINE(key[=value] LF)
101
102 key = 1*(ALPHA | DIGIT | "-_")
103 value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
104
105 Command Request
106-----------------
107
108After receiving the capability advertisement, a client can then issue a
109request to select the command it wants with any particular capabilities
110or arguments. There is then an optional section where the client can
111provide any command specific parameters or queries. Only a single
112command can be requested at a time.
113
114 request = empty-request | command-request
115 empty-request = flush-pkt
116 command-request = command
117 capability-list
118 [command-args]
119 flush-pkt
120 command = PKT-LINE("command=" key LF)
121 command-args = delim-pkt
122 *command-specific-arg
123
124 command-specific-args are packet line framed arguments defined by
125 each individual command.
126
127The server will then check to ensure that the client's request is
128comprised of a valid command as well as valid capabilities which were
129advertised. If the request is valid the server will then execute the
130command. A server MUST wait till it has received the client's entire
131request before issuing a response. The format of the response is
132determined by the command being executed, but in all cases a flush-pkt
133indicates the end of the response.
134
135When a command has finished, and the client has received the entire
136response from the server, a client can either request that another
137command be executed or can terminate the connection. A client may
138optionally send an empty request consisting of just a flush-pkt to
139indicate that no more requests will be made.
140
141 Capabilities
142--------------
143
144There are two different types of capabilities: normal capabilities,
145which can be used to to convey information or alter the behavior of a
146request, and commands, which are the core actions that a client wants to
147perform (fetch, push, etc).
148
149Protocol version 2 is stateless by default. This means that all commands
150must only last a single round and be stateless from the perspective of the
151server side, unless the client has requested a capability indicating that
152state should be maintained by the server. Clients MUST NOT require state
153management on the server side in order to function correctly. This
154permits simple round-robin load-balancing on the server side, without
155needing to worry about state management.
156
157 agent
158~~~~~~~
159
160The server can advertise the `agent` capability with a value `X` (in the
161form `agent=X`) to notify the client that the server is running version
162`X`. The client may optionally send its own agent string by including
163the `agent` capability with a value `Y` (in the form `agent=Y`) in its
164request to the server (but it MUST NOT do so if the server did not
165advertise the agent capability). The `X` and `Y` strings may contain any
166printable ASCII characters except space (i.e., the byte range 32 < x <
167127), and are typically of the form "package/version" (e.g.,
168"git/1.8.3.1"). The agent strings are purely informative for statistics
169and debugging purposes, and MUST NOT be used to programmatically assume
170the presence or absence of particular features.
72d0ea00
BW
171
172 ls-refs
173~~~~~~~~~
174
175`ls-refs` is the command used to request a reference advertisement in v2.
176Unlike the current reference advertisement, ls-refs takes in arguments
177which can be used to limit the refs sent from the server.
178
179Additional features not supported in the base command will be advertised
180as the value of the command in the capability advertisement in the form
181of a space separated list of features: "<command>=<feature 1> <feature 2>"
182
183ls-refs takes in the following arguments:
184
185 symrefs
186 In addition to the object pointed by it, show the underlying ref
187 pointed by it when showing a symbolic ref.
188 peel
189 Show peeled tags.
190 ref-prefix <prefix>
191 When specified, only references having a prefix matching one of
192 the provided prefixes are displayed.
193
194The output of ls-refs is as follows:
195
196 output = *ref
197 flush-pkt
198 ref = PKT-LINE(obj-id SP refname *(SP ref-attribute) LF)
199 ref-attribute = (symref | peeled)
200 symref = "symref-target:" symref-target
201 peeled = "peeled:" obj-id
3145ea95
BW
202
203 fetch
204~~~~~~~
205
206`fetch` is the command used to fetch a packfile in v2. It can be looked
207at as a modified version of the v1 fetch where the ref-advertisement is
208stripped out (since the `ls-refs` command fills that role) and the
209message format is tweaked to eliminate redundancies and permit easy
210addition of future extensions.
211
212Additional features not supported in the base command will be advertised
213as the value of the command in the capability advertisement in the form
214of a space separated list of features: "<command>=<feature 1> <feature 2>"
215
216A `fetch` request can take the following arguments:
217
218 want <oid>
219 Indicates to the server an object which the client wants to
220 retrieve. Wants can be anything and are not limited to
221 advertised objects.
222
223 have <oid>
224 Indicates to the server an object which the client has locally.
225 This allows the server to make a packfile which only contains
226 the objects that the client needs. Multiple 'have' lines can be
227 supplied.
228
229 done
230 Indicates to the server that negotiation should terminate (or
231 not even begin if performing a clone) and that the server should
232 use the information supplied in the request to construct the
233 packfile.
234
235 thin-pack
236 Request that a thin pack be sent, which is a pack with deltas
237 which reference base objects not contained within the pack (but
238 are known to exist at the receiving end). This can reduce the
239 network traffic significantly, but it requires the receiving end
240 to know how to "thicken" these packs by adding the missing bases
241 to the pack.
242
243 no-progress
244 Request that progress information that would normally be sent on
245 side-band channel 2, during the packfile transfer, should not be
246 sent. However, the side-band channel 3 is still used for error
247 responses.
248
249 include-tag
250 Request that annotated tags should be sent if the objects they
251 point to are being sent.
252
253 ofs-delta
254 Indicate that the client understands PACKv2 with delta referring
255 to its base by position in pack rather than by an oid. That is,
256 they can read OBJ_OFS_DELTA (ake type 6) in a packfile.
257
f7e20501
BW
258If the 'shallow' feature is advertised the following arguments can be
259included in the clients request as well as the potential addition of the
260'shallow-info' section in the server's response as explained below.
261
685fbd32
BW
262 shallow <oid>
263 A client must notify the server of all commits for which it only
264 has shallow copies (meaning that it doesn't have the parents of
265 a commit) by supplying a 'shallow <oid>' line for each such
266 object so that the server is aware of the limitations of the
267 client's history. This is so that the server is aware that the
268 client may not have all objects reachable from such commits.
269
270 deepen <depth>
271 Requests that the fetch/clone should be shallow having a commit
272 depth of <depth> relative to the remote side.
273
274 deepen-relative
275 Requests that the semantics of the "deepen" command be changed
276 to indicate that the depth requested is relative to the client's
277 current shallow boundary, instead of relative to the requested
278 commits.
279
280 deepen-since <timestamp>
281 Requests that the shallow clone/fetch should be cut at a
282 specific time, instead of depth. Internally it's equivalent to
283 doing "git rev-list --max-age=<timestamp>". Cannot be used with
284 "deepen".
285
286 deepen-not <rev>
287 Requests that the shallow clone/fetch should be cut at a
288 specific revision specified by '<rev>', instead of a depth.
289 Internally it's equivalent of doing "git rev-list --not <rev>".
290 Cannot be used with "deepen", but can be used with
291 "deepen-since".
292
3145ea95
BW
293The response of `fetch` is broken into a number of sections separated by
294delimiter packets (0001), with each section beginning with its section
295header.
296
297 output = *section
685fbd32 298 section = (acknowledgments | shallow-info | packfile)
3145ea95
BW
299 (flush-pkt | delim-pkt)
300
301 acknowledgments = PKT-LINE("acknowledgments" LF)
302 (nak | *ack)
303 (ready)
304 ready = PKT-LINE("ready" LF)
305 nak = PKT-LINE("NAK" LF)
306 ack = PKT-LINE("ACK" SP obj-id LF)
307
685fbd32
BW
308 shallow-info = PKT-LINE("shallow-info" LF)
309 *PKT-LINE((shallow | unshallow) LF)
310 shallow = "shallow" SP obj-id
311 unshallow = "unshallow" SP obj-id
312
3145ea95
BW
313 packfile = PKT-LINE("packfile" LF)
314 *PKT-LINE(%x01-03 *%x00-ff)
315
316 acknowledgments section
317 * If the client determines that it is finished with negotiations
318 by sending a "done" line, the acknowledgments sections MUST be
319 omitted from the server's response.
320
321 * Always begins with the section header "acknowledgments"
322
323 * The server will respond with "NAK" if none of the object ids sent
324 as have lines were common.
325
326 * The server will respond with "ACK obj-id" for all of the
327 object ids sent as have lines which are common.
328
329 * A response cannot have both "ACK" lines as well as a "NAK"
330 line.
331
332 * The server will respond with a "ready" line indicating that
333 the server has found an acceptable common base and is ready to
334 make and send a packfile (which will be found in the packfile
335 section of the same response)
336
337 * If the server has found a suitable cut point and has decided
338 to send a "ready" line, then the server can decide to (as an
339 optimization) omit any "ACK" lines it would have sent during
340 its response. This is because the server will have already
341 determined the objects it plans to send to the client and no
342 further negotiation is needed.
343
685fbd32 344 shallow-info section
f7e20501
BW
345 * If the client has requested a shallow fetch/clone, a shallow
346 client requests a fetch or the server is shallow then the
347 server's response may include a shallow-info section. The
348 shallow-info section will be included if (due to one of the
349 above conditions) the server needs to inform the client of any
350 shallow boundaries or adjustments to the clients already
351 existing shallow boundaries.
685fbd32
BW
352
353 * Always begins with the section header "shallow-info"
354
355 * If a positive depth is requested, the server will compute the
356 set of commits which are no deeper than the desired depth.
357
358 * The server sends a "shallow obj-id" line for each commit whose
359 parents will not be sent in the following packfile.
360
361 * The server sends an "unshallow obj-id" line for each commit
362 which the client has indicated is shallow, but is no longer
363 shallow as a result of the fetch (due to its parents being
364 sent in the following packfile).
365
366 * The server MUST NOT send any "unshallow" lines for anything
367 which the client has not indicated was shallow as a part of
368 its request.
369
370 * This section is only included if a packfile section is also
371 included in the response.
372
3145ea95
BW
373 packfile section
374 * This section is only included if the client has sent 'want'
375 lines in its request and either requested that no more
376 negotiation be done by sending 'done' or if the server has
377 decided it has found a sufficient cut point to produce a
378 packfile.
379
380 * Always begins with the section header "packfile"
381
382 * The transmission of the packfile begins immediately after the
383 section header
384
385 * The data transfer of the packfile is always multiplexed, using
386 the same semantics of the 'side-band-64k' capability from
387 protocol version 1. This means that each packet, during the
388 packfile data stream, is made up of a leading 4-byte pkt-line
389 length (typical of the pkt-line format), followed by a 1-byte
390 stream code, followed by the actual data.
391
392 The stream code can be one of:
393 1 - pack data
394 2 - progress messages
395 3 - fatal error message just before stream aborts
ecc3e534
BW
396
397 server-option
398~~~~~~~~~~~~~~~
399
400If advertised, indicates that any number of server specific options can be
401included in a request. This is done by sending each option as a
402"server-option=<option>" capability line in the capability-list section of
403a request.
404
405The provided options must not contain a NUL or LF character.