Merge branch 'ls/git-gui-no-double-utf8-author-name'
[git/git.git] / Documentation / technical / http-protocol.txt
1 HTTP transfer protocols
2 =======================
3
4 Git supports two HTTP based transfer protocols. A "dumb" protocol
5 which requires only a standard HTTP server on the server end of the
6 connection, and a "smart" protocol which requires a Git aware CGI
7 (or server module). This document describes both protocols.
8
9 As a design feature smart clients can automatically upgrade "dumb"
10 protocol URLs to smart URLs. This permits all users to have the
11 same published URL, and the peers automatically select the most
12 efficient transport available to them.
13
14
15 URL Format
16 ----------
17
18 URLs for Git repositories accessed by HTTP use the standard HTTP
19 URL syntax documented by RFC 1738, so they are of the form:
20
21 http://<host>:<port>/<path>?<searchpart>
22
23 Within this documentation the placeholder `$GIT_URL` will stand for
24 the http:// repository URL entered by the end-user.
25
26 Servers SHOULD handle all requests to locations matching `$GIT_URL`, as
27 both the "smart" and "dumb" HTTP protocols used by Git operate
28 by appending additional path components onto the end of the user
29 supplied `$GIT_URL` string.
30
31 An example of a dumb client requesting for a loose object:
32
33 $GIT_URL: http://example.com:8080/git/repo.git
34 URL request: http://example.com:8080/git/repo.git/objects/d0/49f6c27a2244e12041955e262a404c7faba355
35
36 An example of a smart request to a catch-all gateway:
37
38 $GIT_URL: http://example.com/daemon.cgi?svc=git&q=
39 URL request: http://example.com/daemon.cgi?svc=git&q=/info/refs&service=git-receive-pack
40
41 An example of a request to a submodule:
42
43 $GIT_URL: http://example.com/git/repo.git/path/submodule.git
44 URL request: http://example.com/git/repo.git/path/submodule.git/info/refs
45
46 Clients MUST strip a trailing `/`, if present, from the user supplied
47 `$GIT_URL` string to prevent empty path tokens (`//`) from appearing
48 in any URL sent to a server. Compatible clients MUST expand
49 `$GIT_URL/info/refs` as `foo/info/refs` and not `foo//info/refs`.
50
51
52 Authentication
53 --------------
54
55 Standard HTTP authentication is used if authentication is required
56 to access a repository, and MAY be configured and enforced by the
57 HTTP server software.
58
59 Because Git repositories are accessed by standard path components
60 server administrators MAY use directory based permissions within
61 their HTTP server to control repository access.
62
63 Clients SHOULD support Basic authentication as described by RFC 2617.
64 Servers SHOULD support Basic authentication by relying upon the
65 HTTP server placed in front of the Git server software.
66
67 Servers SHOULD NOT require HTTP cookies for the purposes of
68 authentication or access control.
69
70 Clients and servers MAY support other common forms of HTTP based
71 authentication, such as Digest authentication.
72
73
74 SSL
75 ---
76
77 Clients and servers SHOULD support SSL, particularly to protect
78 passwords when relying on Basic HTTP authentication.
79
80
81 Session State
82 -------------
83
84 The Git over HTTP protocol (much like HTTP itself) is stateless
85 from the perspective of the HTTP server side. All state MUST be
86 retained and managed by the client process. This permits simple
87 round-robin load-balancing on the server side, without needing to
88 worry about state management.
89
90 Clients MUST NOT require state management on the server side in
91 order to function correctly.
92
93 Servers MUST NOT require HTTP cookies in order to function correctly.
94 Clients MAY store and forward HTTP cookies during request processing
95 as described by RFC 2616 (HTTP/1.1). Servers SHOULD ignore any
96 cookies sent by a client.
97
98
99 General Request Processing
100 --------------------------
101
102 Except where noted, all standard HTTP behavior SHOULD be assumed
103 by both client and server. This includes (but is not necessarily
104 limited to):
105
106 If there is no repository at `$GIT_URL`, or the resource pointed to by a
107 location matching `$GIT_URL` does not exist, the server MUST NOT respond
108 with `200 OK` response. A server SHOULD respond with
109 `404 Not Found`, `410 Gone`, or any other suitable HTTP status code
110 which does not imply the resource exists as requested.
111
112 If there is a repository at `$GIT_URL`, but access is not currently
113 permitted, the server MUST respond with the `403 Forbidden` HTTP
114 status code.
115
116 Servers SHOULD support both HTTP 1.0 and HTTP 1.1.
117 Servers SHOULD support chunked encoding for both request and response
118 bodies.
119
120 Clients SHOULD support both HTTP 1.0 and HTTP 1.1.
121 Clients SHOULD support chunked encoding for both request and response
122 bodies.
123
124 Servers MAY return ETag and/or Last-Modified headers.
125
126 Clients MAY revalidate cached entities by including If-Modified-Since
127 and/or If-None-Match request headers.
128
129 Servers MAY return `304 Not Modified` if the relevant headers appear
130 in the request and the entity has not changed. Clients MUST treat
131 `304 Not Modified` identical to `200 OK` by reusing the cached entity.
132
133 Clients MAY reuse a cached entity without revalidation if the
134 Cache-Control and/or Expires header permits caching. Clients and
135 servers MUST follow RFC 2616 for cache controls.
136
137
138 Discovering References
139 ----------------------
140
141 All HTTP clients MUST begin either a fetch or a push exchange by
142 discovering the references available on the remote repository.
143
144 Dumb Clients
145 ~~~~~~~~~~~~
146
147 HTTP clients that only support the "dumb" protocol MUST discover
148 references by making a request for the special info/refs file of
149 the repository.
150
151 Dumb HTTP clients MUST make a `GET` request to `$GIT_URL/info/refs`,
152 without any search/query parameters.
153
154 C: GET $GIT_URL/info/refs HTTP/1.0
155
156 S: 200 OK
157 S:
158 S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint
159 S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master
160 S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0
161 S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}
162
163 The Content-Type of the returned info/refs entity SHOULD be
164 `text/plain; charset=utf-8`, but MAY be any content type.
165 Clients MUST NOT attempt to validate the returned Content-Type.
166 Dumb servers MUST NOT return a return type starting with
167 `application/x-git-`.
168
169 Cache-Control headers MAY be returned to disable caching of the
170 returned entity.
171
172 When examining the response clients SHOULD only examine the HTTP
173 status code. Valid responses are `200 OK`, or `304 Not Modified`.
174
175 The returned content is a UNIX formatted text file describing
176 each ref and its known value. The file SHOULD be sorted by name
177 according to the C locale ordering. The file SHOULD NOT include
178 the default ref named `HEAD`.
179
180 info_refs = *( ref_record )
181 ref_record = any_ref / peeled_ref
182
183 any_ref = obj-id HTAB refname LF
184 peeled_ref = obj-id HTAB refname LF
185 obj-id HTAB refname "^{}" LF
186
187 Smart Clients
188 ~~~~~~~~~~~~~
189
190 HTTP clients that support the "smart" protocol (or both the
191 "smart" and "dumb" protocols) MUST discover references by making
192 a parameterized request for the info/refs file of the repository.
193
194 The request MUST contain exactly one query parameter,
195 `service=$servicename`, where `$servicename` MUST be the service
196 name the client wishes to contact to complete the operation.
197 The request MUST NOT contain additional query parameters.
198
199 C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
200
201 dumb server reply:
202
203 S: 200 OK
204 S:
205 S: 95dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint
206 S: d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master
207 S: 2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0
208 S: a3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}
209
210 smart server reply:
211
212 S: 200 OK
213 S: Content-Type: application/x-git-upload-pack-advertisement
214 S: Cache-Control: no-cache
215 S:
216 S: 001e# service=git-upload-pack\n
217 S: 004895dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0multi_ack\n
218 S: 0042d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n
219 S: 003c2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n
220 S: 003fa3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n
221
222 The client may send Extra Parameters (see
223 Documentation/technical/pack-protocol.txt) as a colon-separated string
224 in the Git-Protocol HTTP header.
225
226 Dumb Server Response
227 ^^^^^^^^^^^^^^^^^^^^
228 Dumb servers MUST respond with the dumb server reply format.
229
230 See the prior section under dumb clients for a more detailed
231 description of the dumb server response.
232
233 Smart Server Response
234 ^^^^^^^^^^^^^^^^^^^^^
235 If the server does not recognize the requested service name, or the
236 requested service name has been disabled by the server administrator,
237 the server MUST respond with the `403 Forbidden` HTTP status code.
238
239 Otherwise, smart servers MUST respond with the smart server reply
240 format for the requested service name.
241
242 Cache-Control headers SHOULD be used to disable caching of the
243 returned entity.
244
245 The Content-Type MUST be `application/x-$servicename-advertisement`.
246 Clients SHOULD fall back to the dumb protocol if another content
247 type is returned. When falling back to the dumb protocol clients
248 SHOULD NOT make an additional request to `$GIT_URL/info/refs`, but
249 instead SHOULD use the response already in hand. Clients MUST NOT
250 continue if they do not support the dumb protocol.
251
252 Clients MUST validate the status code is either `200 OK` or
253 `304 Not Modified`.
254
255 Clients MUST validate the first five bytes of the response entity
256 matches the regex `^[0-9a-f]{4}#`. If this test fails, clients
257 MUST NOT continue.
258
259 Clients MUST parse the entire response as a sequence of pkt-line
260 records.
261
262 Clients MUST verify the first pkt-line is `# service=$servicename`.
263 Servers MUST set $servicename to be the request parameter value.
264 Servers SHOULD include an LF at the end of this line.
265 Clients MUST ignore an LF at the end of the line.
266
267 Servers MUST terminate the response with the magic `0000` end
268 pkt-line marker.
269
270 The returned response is a pkt-line stream describing each ref and
271 its known value. The stream SHOULD be sorted by name according to
272 the C locale ordering. The stream SHOULD include the default ref
273 named `HEAD` as the first ref. The stream MUST include capability
274 declarations behind a NUL on the first ref.
275
276 The returned response contains "version 1" if "version=1" was sent as an
277 Extra Parameter.
278
279 smart_reply = PKT-LINE("# service=$servicename" LF)
280 *1("version 1")
281 ref_list
282 "0000"
283 ref_list = empty_list / non_empty_list
284
285 empty_list = PKT-LINE(zero-id SP "capabilities^{}" NUL cap-list LF)
286
287 non_empty_list = PKT-LINE(obj-id SP name NUL cap_list LF)
288 *ref_record
289
290 cap-list = capability *(SP capability)
291 capability = 1*(LC_ALPHA / DIGIT / "-" / "_")
292 LC_ALPHA = %x61-7A
293
294 ref_record = any_ref / peeled_ref
295 any_ref = PKT-LINE(obj-id SP name LF)
296 peeled_ref = PKT-LINE(obj-id SP name LF)
297 PKT-LINE(obj-id SP name "^{}" LF
298
299
300 Smart Service git-upload-pack
301 ------------------------------
302 This service reads from the repository pointed to by `$GIT_URL`.
303
304 Clients MUST first perform ref discovery with
305 `$GIT_URL/info/refs?service=git-upload-pack`.
306
307 C: POST $GIT_URL/git-upload-pack HTTP/1.0
308 C: Content-Type: application/x-git-upload-pack-request
309 C:
310 C: 0032want 0a53e9ddeaddad63ad106860237bbf53411d11a7\n
311 C: 0032have 441b40d833fdfa93eb2908e52742248faf0ee993\n
312 C: 0000
313
314 S: 200 OK
315 S: Content-Type: application/x-git-upload-pack-result
316 S: Cache-Control: no-cache
317 S:
318 S: ....ACK %s, continue
319 S: ....NAK
320
321 Clients MUST NOT reuse or revalidate a cached response.
322 Servers MUST include sufficient Cache-Control headers
323 to prevent caching of the response.
324
325 Servers SHOULD support all capabilities defined here.
326
327 Clients MUST send at least one "want" command in the request body.
328 Clients MUST NOT reference an id in a "want" command which did not
329 appear in the response obtained through ref discovery unless the
330 server advertises capability `allow-tip-sha1-in-want` or
331 `allow-reachable-sha1-in-want`.
332
333 compute_request = want_list
334 have_list
335 request_end
336 request_end = "0000" / "done"
337
338 want_list = PKT-LINE(want NUL cap_list LF)
339 *(want_pkt)
340 want_pkt = PKT-LINE(want LF)
341 want = "want" SP id
342 cap_list = *(SP capability) SP
343
344 have_list = *PKT-LINE("have" SP id LF)
345
346 TODO: Document this further.
347
348 The Negotiation Algorithm
349 ~~~~~~~~~~~~~~~~~~~~~~~~~
350 The computation to select the minimal pack proceeds as follows
351 (C = client, S = server):
352
353 'init step:'
354
355 C: Use ref discovery to obtain the advertised refs.
356
357 C: Place any object seen into set `advertised`.
358
359 C: Build an empty set, `common`, to hold the objects that are later
360 determined to be on both ends.
361
362 C: Build a set, `want`, of the objects from `advertised` the client
363 wants to fetch, based on what it saw during ref discovery.
364
365 C: Start a queue, `c_pending`, ordered by commit time (popping newest
366 first). Add all client refs. When a commit is popped from
367 the queue its parents SHOULD be automatically inserted back.
368 Commits MUST only enter the queue once.
369
370 'one compute step:'
371
372 C: Send one `$GIT_URL/git-upload-pack` request:
373
374 C: 0032want <want #1>...............................
375 C: 0032want <want #2>...............................
376 ....
377 C: 0032have <common #1>.............................
378 C: 0032have <common #2>.............................
379 ....
380 C: 0032have <have #1>...............................
381 C: 0032have <have #2>...............................
382 ....
383 C: 0000
384
385 The stream is organized into "commands", with each command
386 appearing by itself in a pkt-line. Within a command line,
387 the text leading up to the first space is the command name,
388 and the remainder of the line to the first LF is the value.
389 Command lines are terminated with an LF as the last byte of
390 the pkt-line value.
391
392 Commands MUST appear in the following order, if they appear
393 at all in the request stream:
394
395 * "want"
396 * "have"
397
398 The stream is terminated by a pkt-line flush (`0000`).
399
400 A single "want" or "have" command MUST have one hex formatted
401 SHA-1 as its value. Multiple SHA-1s MUST be sent by sending
402 multiple commands.
403
404 The `have` list is created by popping the first 32 commits
405 from `c_pending`. Less can be supplied if `c_pending` empties.
406
407 If the client has sent 256 "have" commits and has not yet
408 received one of those back from `s_common`, or the client has
409 emptied `c_pending` it SHOULD include a "done" command to let
410 the server know it won't proceed:
411
412 C: 0009done
413
414 S: Parse the git-upload-pack request:
415
416 Verify all objects in `want` are directly reachable from refs.
417
418 The server MAY walk backwards through history or through
419 the reflog to permit slightly stale requests.
420
421 If no "want" objects are received, send an error:
422 TODO: Define error if no "want" lines are requested.
423
424 If any "want" object is not reachable, send an error:
425 TODO: Define error if an invalid "want" is requested.
426
427 Create an empty list, `s_common`.
428
429 If "have" was sent:
430
431 Loop through the objects in the order supplied by the client.
432
433 For each object, if the server has the object reachable from
434 a ref, add it to `s_common`. If a commit is added to `s_common`,
435 do not add any ancestors, even if they also appear in `have`.
436
437 S: Send the git-upload-pack response:
438
439 If the server has found a closed set of objects to pack or the
440 request ends with "done", it replies with the pack.
441 TODO: Document the pack based response
442
443 S: PACK...
444
445 The returned stream is the side-band-64k protocol supported
446 by the git-upload-pack service, and the pack is embedded into
447 stream 1. Progress messages from the server side MAY appear
448 in stream 2.
449
450 Here a "closed set of objects" is defined to have at least
451 one path from every "want" to at least one "common" object.
452
453 If the server needs more information, it replies with a
454 status continue response:
455 TODO: Document the non-pack response
456
457 C: Parse the upload-pack response:
458 TODO: Document parsing response
459
460 'Do another compute step.'
461
462
463 Smart Service git-receive-pack
464 ------------------------------
465 This service reads from the repository pointed to by `$GIT_URL`.
466
467 Clients MUST first perform ref discovery with
468 `$GIT_URL/info/refs?service=git-receive-pack`.
469
470 C: POST $GIT_URL/git-receive-pack HTTP/1.0
471 C: Content-Type: application/x-git-receive-pack-request
472 C:
473 C: ....0a53e9ddeaddad63ad106860237bbf53411d11a7 441b40d833fdfa93eb2908e52742248faf0ee993 refs/heads/maint\0 report-status
474 C: 0000
475 C: PACK....
476
477 S: 200 OK
478 S: Content-Type: application/x-git-receive-pack-result
479 S: Cache-Control: no-cache
480 S:
481 S: ....
482
483 Clients MUST NOT reuse or revalidate a cached response.
484 Servers MUST include sufficient Cache-Control headers
485 to prevent caching of the response.
486
487 Servers SHOULD support all capabilities defined here.
488
489 Clients MUST send at least one command in the request body.
490 Within the command portion of the request body clients SHOULD send
491 the id obtained through ref discovery as old_id.
492
493 update_request = command_list
494 "PACK" <binary data>
495
496 command_list = PKT-LINE(command NUL cap_list LF)
497 *(command_pkt)
498 command_pkt = PKT-LINE(command LF)
499 cap_list = *(SP capability) SP
500
501 command = create / delete / update
502 create = zero-id SP new_id SP name
503 delete = old_id SP zero-id SP name
504 update = old_id SP new_id SP name
505
506 TODO: Document this further.
507
508
509 References
510 ----------
511
512 http://www.ietf.org/rfc/rfc1738.txt[RFC 1738: Uniform Resource Locators (URL)]
513 http://www.ietf.org/rfc/rfc2616.txt[RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1]
514 link:technical/pack-protocol.html
515 link:technical/protocol-capabilities.html