Close

Sign up to Gadget

Sign Up

File Storage with the File field type

Harry Brundage
December 13, 2021

From user profile images to invoice PDFs to cute dog pics, it’s hard to find a modern application that doesn’t need to store files. We’re excited to announce that as of today, every Gadget application can store and serve files in the cloud, with zero configuration. Gadget accepts uploads in a variety of formats, stores file contents securely in Google Cloud Storage, and serves files using a performant CDN.

Here’s how it works:

  • You add a new field to any model, including those from a Connection, with the File field type
  • You use your app’s API client, or a GraphQL call to upload a file as the value for this new field on a record
  • Your app’s API writes the file to a private, cloud storage bucket that we’ve setup for you
  • When you read the record, Gadget returns a URL to access the file securely

There’s no time spent setting up cloud buckets, configuring security rules, or dealing with cross origin headers -- just add a File field and you’re set to go.

The File field type supports validations just like any other field, so you can limit the size or nature of files your app will store. File fields can be edited or changed within effect code snippets along with the rest of the record as well. And finally, you can limit file access using the same role-based permission-ing system that handles access to other fields and records.

We’ve also tried to make it as easy as possible to write frontend code to create files in Gadget. You can submit files to your Gadget app’s API using base64 encoded strings, using multipart form requests from the browser, or using an existing URL to a file. And finally, Gadget supports direct upload to cloud storage, which gives a great user experience when uploading bigger files. Your generated Gadget API client already knows how to communicate with the Gadget API for all these methods, so uploading files works just the same as any other action.

You can see an example of a React application which uploads files to Gadget here and read the File field docs here.

For more examples on what you can build using the Shopify Connection, check out these additional blog posts:

Keep reading to learn about how it's built

Under the hood

Uploading and serving files has been something many folks have had to sort out many times over in the past. But, it can be really annoying to flesh out this feature with the extra security and performance that many customers have begun to expect. With the File field, we've tried to give both users and developers a best-in-class experience right out of the box.

For the user experience, we focused on making building excellent web forms easy, and a big part of that is giving users feedback as they fill in form's fields. Users are able to correct mistakes quickly right after making them, instead of filling out the entire form, hitting save, and having to scroll around and fix various things they’ve already forgotten touching. File uploads pose a similarly dangerous user experience challenge where they can slow down the cycle of a user entering data, submitting it, waiting for the files to upload and the data to be validated, and then learning that they got something wrong and having to do it again.

Instead, we should try to get users through their forms as fast as possible, and one great way to do that is to start uploading any files as soon as they select them, instead of waiting for them to hit save to begin. That way, while they are filling out any other fields, the upload can progress in the background and hopefully complete before they’re finished. The uploaded file can also be re-used between different form submissions, so that if other fields happen to be invalid on the form, the file doesn’t need to be uploaded a second time.

The process works like this:

  1. The user selects a file in an <inline-code><input type=”file”/><inline-code> or similar 
  2. The <inline-code>onchange<inline-code> event handlers reaches out to Gadget to provision a one-time-use direct upload url and token, and begins POSTing the file to that URL immediately
  3. The form `submit` event handler uses the returned token as the value for the File field, instead of the file’s contents
  4. When the user submits the form, Gadget receives the token and moves the file in cloud storage to the correct location.

This approach is called “direct uploads” and gives the best user experience. It requires a bit fancier server that knows how to create these direct upload URL targets and tokens as well as a bit fancier client that knows how to work with these files. Gadget makes this easier by already having the server, cloud storage bucket, and secure URL signing set up for you. We use Google Cloud Storage’s signed upload URL functionality to let user browsers upload files right into cloud storage which performs way better than us proxying it. We also use a different bucket for direct uploads so we can secure it differently, and so we can garbage collect unconsumed directly uploaded files easily.


We’ve got two examples of direct upload frontends to help you get started. There’s a raw React example here, and one using an amazing open source file picker widget called Uppy here. And as always, if you have any questions or could use some help, come talk to us on Discord!

We're on Discord, drop in and say hi!
Join Discord
Bouncing Arrow