TLDR: See changes to your database as they happen, and get low-latency notifications to keep your frontend updated using realtime queries.
Modern apps need instant access to timely and accurate information. With the constant flux of data in web apps, users expect real-time updates without having to refresh their pages. This introduces a challenge for developers: we need a way to ensure our apps are always up-to-date without compromising on performance.
Enter Gadget's new realtime queries, an easy way to ensure your frontend is able to reflect backend changes instantly.
Without realtime data, developers are required to set up workarounds like polling the database on a schedule to see if data has changed over time, a method that tends to be slow and unreliable due to the limitations of those requests. Depending on the window you choose for polling your database, there’s always a risk that the data your end users are seeing is out of date.
Polling your database every few seconds also quickly becomes expensive, as some server somewhere needs to execute that query over and over. Without realtime queries, not only do you run the risk of delivering outdated information, but you have to pay for it too!
Ditch the traditional change polling and refresh buttons. With a one line change, you can now subscribe to your backend via WebSocket, and instantly update your UI as data changes. You can show the latest changes to users instantly, because any record creation, update, or deletion in the backend will trigger an immediate re-fetch of data in the frontend.
To turn on live querying, pass the <inline-code>live: true<inline-code> option to the hooks from the <inline-code>@gadgetinc/react<inline-code> library:
Realtime queries are supported in <inline-code>@gadgetinc/react<inline-code> as of version 0.14.9. Existing Gadget users can update their version using the Command Palette by pressing CMD+P, then entering > to start the command mode, then entering: <inline-code>yarn upgrade @gadgetinc/react@latest<inline-code> to upgrade their package version.
Gadget's realtime queries also allow you to dive deeper into nested data. For instance, if you select related records in your queries, changes made to those records will also push updates and trigger re-renders in your component. You can also combine live queries with the select option to specify which fields you want to render, making sure you're always presenting the most relevant, up-to-date information to your users.
Realtime queries are available on both single records and collections of records in Gadget.
To get started, all you need to do is add the live: true option to one of the supported React hooks For example, we could render an up-to-date view of a user’s top 5 most recent unfinished todos in a todo application:
Once this is in place, any relevant changes to the backend to-dos will trigger a re-render of your frontend components, providing an always up-to-date view. This includes new to-dos being created, to-dos getting marked as completed, or to-dos being deleted.
Realtime queries are a necessity in modern app building. With Gadget's out-of-the-box support, building dynamic and real-time applications has never been easier.
If you have questions about using realtime queries in your next app build you can read about it in our docs, or join the Gadget Discord to connect with other Shopify app developers.
Gadget’s realtime query support uses the same rich, autogenerated GraphQL API that powers every app on the platform. Out of the box, GraphQL has support for long-running subscription requests to the backend in the form of the subscriptions endpoint. Subscriptions describe the changes in data at too low a level because they often just return a stream of events, instead of a fully assembled, ready-to-render graph of data. Instead of receiving new data to render, you receive a bulk of change events describing many different records, with different fields and filters, which you must transform into the data you want to render.
So we asked ourselves, “Why should realtime queries have to use a totally different data access style?”
Our answer: “We don’t think they should!”
Gadget’s realtime queries are built on top of the GraphQL <inline-code>@live<inline-code> directive. <inline-code>@live<inline-code> queries fetch the same data a non-realtime query would, and then stream the exact changes to that data structure from the backend. This is highly efficient as the JSON diffs are small and take advantage of data already on the client. You don’t need to manage any callbacks or handlers, you don’t need to worry about transforming events or caching, and you get great responsiveness as you only retrieve data that has truly changed, instead of a notice that something might have changed. For more on the difference between @live queries and subscriptions, see this great breakdown by The Guild.
We also took on the challenge of building a scalable backend implementation for each app’s <inline-code>@live<inline-code> API. Internally, Gadget supports a variety of storage adapters for database data – a variety of Postgres-based storage adapters for durable data, and a Redis storage adapter for ephemeral data-like sessions. Each storage adapter in Gadget publishes each change to the database to a shared internal event bus. The @live query backend then subscribes to this event bus, watching for any events concerning records already returned by the query, or any events on records that might change the records on the current page. This is tricky to get right, as you need to observe the database set that could be included in the result if a change happened, instead of just what is currently on screen. In our todo example above, we’re only showing uncompleted todos on the dashboard, but if an already-completed todo gets marked as incomplete, it needs to bump one of the other ones off the list and show up.
Instead of trying to monitor thousands upon thousands of individual record IDs, or worse the entire database, <inline-code>@live<inline-code> query backends instead watch for particular record attributes on the stream. When any relevant records are seen, Gadget will re-execute the database query that powered the initial result set. This means you get the most up-to-date information that has all the records that may now be within the set, including those that came onto or off the page due to filters or pagination criteria. Gadget then compares this new set against the old set and emits the difference to the client.
Building the live query system has been a fun journey, and we hope you find it as productive as we do!
