Gadget simplifies the process of sending Shopify data to an ERP or other existing systems. Integrating with Shopify can be challenging, which is why many builders use Gadget as an in-between step. Whether you need to synchronize data with Shopify, respond to webhooks, or handle authorization, Gadget can serve as middleware to facilitate the seamless movement of data between services.
In this guide, we will use Gadget to build a middleware application that forwards new orders from Shopify to an existing service. Keep in mind that existing services may have certain restrictions, such as rate limits, that need to be carefully managed. As an example, we will utilize Google Cloud Tasks to demonstrate how to queue requests and payloads for forwarding to an existing service.
Requirements
Before you begin this guide, make sure you have the following:
→ a Shopify Partners account
→ a development store
→ a Google Cloud account
→ some familiarity with Actions in Gadget. If you are new to Gadget, start with the automated product tagger tutorial
You can fork this Gadget project and try it out yourself.
After forking, you will still need to:
Our first step will be to set up a Gadget project and connect our backend to a Shopify store via the Shopify connection. Create a new Gadget application at gadget.new and select the Shopify app template.
To connect our Gadget backend to a Shopify store, you have two options. The recommended approach is to create a custom app via the Shopify Partners dashboard. Alternatively, you can create a custom application on your Shopify store Admin page. Both methods involve slightly different workflows for establishing the connection, which is detailed below.
Both the Shopify store Admin and the Shopify Partner Dashboard have an Apps section. Ensure that you are on the Shopify Partner Dashboard before continuing.
Now we get to select what Shopify scopes we give our application access to, while also picking what Shopify data models we want to import into our Gadget app.
Need different scopes or models?
This example uses the Order scope and model, but you can select any Shopify model that you want to import into Gadget. A connection can be edited after it is created, so you can always change scopes and models later.
Now we want to connect our Gadget app to our custom app in the Partners dashboard.
Protected customer data access
If you are using any of the data models that require access to protected customer data, such as Orders, you need to request access to this data from Shopify. To do this, you can:
→ Find the Protected customer data access section on the App setup page of your Shopify Partners dashboard
→ Click Request access and follow the instructions
Once you have gone through this process, you will be able to properly register webhooks and sync data for the protected customer data models.
For information, such as the models requiring you to fill out the protected customer data access form in Shopify, see our documentation.
Gadget will copy the selected Shopify Order models, their types, validations, and associations into your Gadget backend. These models are ready to process webhooks as soon as you install the app on a Shopify store.
Google Cloud Tasks is a fully managed service that facilitates the management, execution, dispatch, and delivery of a large number of distributed tasks. Cloud Tasks will be utilized to queue requests and payloads for forwarding to an existing service.
Before proceeding, ensure that you already have a Google Cloud account. If not, you can create one here. Note that using Cloud Tasks will require you to provide billing information.
You can set up a Cloud Tasks queue using Google's Cloud Console or the gcloud CLI. This guide uses the Cloud Console to set up a Cloud Tasks queue.
Not worried about rate limits on your existing service?
If you aren't worried about rate limits or other restrictions on pushing data to your existing service then you don't need to use Google Cloud Tasks. Instead, you can add a code effect that simply forwards data to your existing service:
If you don't already have a Google Cloud project, you will need to create one, enable the Cloud Tasks API, and create a queue:
Queue settings can be edited at a later time and will be unique to your use case. See Google's docs on queue configuration for more information on the available routing, rate limit, and retry settings.
Now that you have a queue, you can set up a Google service account to handle HTTP Target authentication. This service account will be used to add credentials to the Cloud Tasks API. You can follow Google's instructions, or do the following:
Once you have created your service account, you can generate a key that will be used to authenticate the client used to enqueue tasks from Gadget.
A .json file containing the private key should be downloaded on your local machine. You will need to add the private key to your Gadget app as an environment variable.
Now that a Google Cloud Tasks queue has been set up, it is time to set up the client in Gadget so you can push tasks to the queue. You will need to add the following environment variables to your Gadget app:
These variables will all be used to create tasks that will be pushed to the queue. In this example, a RequestBin is used to receive the tasks and respond with a 200 status code.
Before adding any code in Gadget, you can add the Google Cloud Tasks Node client to your Gadget project.
The @google-cloud/tasks dependency will be added to your Gadget project's package.json file.
Now you can make use of the client in some custom code.
With the connection to a Shopify store and the Google Cloud Tasks queue in place, you can add some custom code to forward any new orders to our existing service. You can do this by creating new tasks that are pushed to the Cloud Tasks queue. Once a task is pushed to the queue, Google Cloud Tasks will handle the rest, including retries and rate limits. The steps to add a success effect and code file are the same for other imported Shopify models if you are not using orders.
The following snippet will forward new order information to an existing backend service that was also built using Gadget. You will get a simple response with the id of the order that was forwarded. The trigger param that is used here contains the entire webhook payload from Shopify.
If you are using shopifyOrder data, you can make a test order on your development store to test your app. If not, complete an action that triggers a webhook to fire for your Shopify data model of interest. You should see a new task in the Google Cloud Tasks queue. The default example above has a 60-second delay between the task being added to the queue and the task being executed. You can change this by modifying the inSeconds variable in the code snippet. After the delay, you should see a new request made against your existing service or RequestBin.
Need authentication for your existing service?
Google Cloud Tasks can also be configured to handle authentication on the existing service. See the Google Cloud Tasks docs for more information.
If you also want to run this code on an update webhook from Shopify:
Now order webhooks will be queued and forwarded to your existing service when an order is created or updated. You can also add code snippets to other actions on the Shopify models, such as delete.
Some things to remember before deploying to Production:
Deploying your application to Production can be done in two clicks:
That is all that is required to use Gadget as middleware. In this example, Gadget gives you hands-off data sync and webhook handling with the Shopify connection, while allowing you to continue to use your existing service to manage your store's data.
If you have any questions about this guide, feel free to reach out on Gadget's developer Discord.