Skip to content
This repository has been archived by the owner on May 15, 2024. It is now read-only.

Commit

Permalink
Signal content length to upstream components when present
Browse files Browse the repository at this point in the history
When the HTTP request contains `Content-Length` header, include it in
the body reader passed down to upstream system components. This is to
enable further optimisations during processing when the total size of a
blob is known beforehand. Examples include piece selection in order to
guarantee that a blob will be fully contained within a piece.

With changes introduced, an upstream component may type-check for
`interface { Size() int64}` and use it to then gain access to the
content length.

Note that based on HTTP standards, `application/octet-stream` content
type is not strictly required to specify `Content-Length` header. Hence
the opportunistic signalling via type-checkers.
  • Loading branch information
masih committed Aug 8, 2023
1 parent 474f315 commit 91f6899
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 2 deletions.
10 changes: 8 additions & 2 deletions api/server/handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ func (m *HttpServer) handlePostBlob(w http.ResponseWriter, r *http.Request) {
respondWithJson(w, errResponseNotStreamContentType, http.StatusBadRequest)
return
}
body := r.Body
var contentLength uint64
if value := r.Header.Get("Content-Length"); value != "" {
var err error
Expand All @@ -38,9 +39,14 @@ func (m *HttpServer) handlePostBlob(w http.ResponseWriter, r *http.Request) {
respondWithJson(w, errResponseContentLengthTooLarge(m.maxBlobLength), http.StatusBadRequest)
return
}
// Wrap body reader to signal content length to upstream components.
body = sizerReadCloser{
ReadCloser: r.Body,
size: int64(contentLength),
}
}
defer r.Body.Close()
desc, err := m.store.Put(r.Context(), r.Body)
defer body.Close()
desc, err := m.store.Put(r.Context(), body)
switch err {
case nil:
case blob.ErrBlobTooLarge:
Expand Down
14 changes: 14 additions & 0 deletions api/server/sizer_reader.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package server

import "io"

var _ interface{ Size() int64 } = (*sizerReadCloser)(nil)

type sizerReadCloser struct {
io.ReadCloser
size int64
}

func (s sizerReadCloser) Size() int64 {
return s.size
}

0 comments on commit 91f6899

Please sign in to comment.