Direct to S3 Uploads

Direct to S3 Uploads

Certain cloud storage platforms, such as Amazon S3, allow for clients to upload files directly to their servers, rather than proxying through your application layer.

This has many advantages, including:

• Bandwidth and load taken away from your application layer.
• Your file streaming behavior is not as efficient as Amazon S3's.
• Clients can upload arbitrarily large files (though you retain control!)

Setup

First, you must create an Amazon S3 bucket and configure both ExAws and Arc.

CORS-Enabled POST

If you're building a JavaScript-enabled application, you may upload files via CORS to avoid redirection / full page loads.

To do so, you must whitelist POST operations to your bucket:

This will allow the client's browser to submit files, and subsequently act upon successful file uploads without any full-page submissions and/or redirections.

Basic File Uploading

Arc's S3 Storage adapter must generate the form fields for uploading to S3:

form = Arc.Storage.S3.html_upload_form() #=> %{action: String.t, meta: Map.t, fields: Map.t}

This form has many options, which you can view here: (TODO)

To submit via Elixir:

image_path = "/var/tmp/image.png"

multipart_fields =
  form.fields
  |> Map.to_list()
  |> Kernel.++([{:file, image_path}])

{:ok, response} = HTTPoison.post(form.action, {:multipart, multipart_fields})

To submit via Phoenix Forms:

    <% form = Arc.Storage.S3.html_upload_form() %>

    <%= form_tag(form.action, method: :post, multipart: true, csrf_token: false, enforce_utf8: false) do %>
      <%= for {name, value} <- form.fields do %>
        <%= tag(:input, type: "hidden", name: name, value: value) %>
      <% end %>

      <%= tag(:input, type: "file", name: "file") %>
      <br/>
      <%= submit("Submit") %>
    <% end %>

Uploaded file URLs

Once a file has been uploaded to S3 directly, it is up to your implementation to act upon that information. In general, there's a couple directions you may choose to go:

• Store the file key directly in a column on a record.
• Send the uploaded file through an Arc transformation.

Generating File URLs

Arc's S3 Storage adapter provides some convenience methods for generating urls of keys. If you upload files to Amazon S3 as private files, then all HTTP file access must be done through signed urls.

s3_key = "uploads/b1cfef0d-a83f-473f-960d-c37f9de2f777"

Arc.Storage.S3.url(s3_key) #=> generates a public url
Arc.Storage.S3.url(s3_key, signed: true) #=> generates a private, signed url

Transformations

After a file is uploaded to S3, you may treat it the exact same as any other remote file:

s3_key = "uploads/b1cfef0d-a83f-473f-960d-c37f9de2f777"
signed_url = Arc.Storage.S3.url(s3_key, signed: true) #=> generates a private, signed url
{:ok, filename} = Arc.store({signed_url, scope})

Removing Files

If you would like to remove an uploaded file from S3, you may:

s3_key = "uploads/b1cfef0d-a83f-473f-960d-c37f9de2f777"
Arc.Storage.S3.delete(s3_key)