by There are a lot of articles out there talking about how great it is to use OTA updates in production, but in this article, I’d like to focus on the impact OTA updates can have during development and testing your React Native applications. In a previous project I worked on, we used OTA updates in staging to speed up the development-to-QA process. In this article I’ll explore some of the problems we faced in making new features accessible for QAs and how OTA updates helped us to achieve a more efficient process.
OTA update, or Over-The-Air update is a method to push code updates to users without the need of creating new builds. In this article, I will be referring to the Expo solution for OTA updates, the EAS update. OTA updates can be used to push changes to the Javascript part of the app and these updates are auto-downloaded by the builds when users start the app. Making it great to push bug fixes and small changes, while also making the update process very similar to the web, where changes are deployed and users just need to refresh the page to use the latest version of your app.
In production, another benefit of OTA updates is that you can push updates without having to wait for your app to pass through the app review process. Making it ideal for shipping urgent bug fixes.
A regular mobile development process
Normally, developers will work on new features, and when these features are ready, ideally, they will be sent to QA. If you are familiar with web development, you would argue that this is a trivial thing to do, right? However, mobile app deployments require a little bit more thought. To send new features to QA, you will need to create a staging build of your app, and this is a process that can be time-consuming. After that, you will need to distribute your build and make it available for QA engineers to download.
In our project, some builds would take up to 40 minutes to complete. If you are developing for both Android and iOS, you should consider that you will need to run separate build pipelines for each platform. Expo has a service to run the build process on the cloud called EAS build. At the time I’m writing this article, they are charging $2 per medium iOS build and $1 per medium Android build. I believe their charges are reasonable, but consider how often you will be creating new builds to avoid unexpected fees.
Check out the EAS pricing options here
Internal distribution
Once you have the builds for Android and iOS, you will need to distribute them to QAs. In my experience, the easiest way to distribute iOS builds, is using TestFlight. QAs will need to download the TestFlight app and receive an invite to test your app. After that, the app developers can submit builds to TestFlight and all users invited to the project will be able to download the app.
For Android, the simplest method is to share APK bundle files with your team. If you want something similar to TestFlight, you can submit your app to the Google Play store and distribute it on the internal testing track. This track is visible only to invited users and your app is not required to pass through the app review process.
How EAS update make this process different?
With OTA you will still have to go through this process of creating and distributing builds for testing, but you will need to do it only when adding changes to native code. These changes may happen when you install a new library to access resources such as camera or geolocation, or when you need to upgrade the Expo SDK. Over time, most projects tend to settle on a few libraries and most changes are just Javascript, making these changes eligible to be deployed through OTA updates.
In a workflow without OTA updates, you would have to go through the process of creating and distributing new builds every time you finish a task. Sometimes people will accumulate features to deploy them in just one deployment. While this approach works fine for production, we experienced firsthand that this compromised a lot of the velocity of how tasks were tested.
What OTA updates bring to the table is the possibility of tasks that involve non-native changes to be deployed to staging as soon they pass code review and are ready for QA.
The Problem
In teams with multiple mobile developers working on the same application, it is common for not everybody to have access to publish builds and tasks can pile up waiting for the next deployment. That was exactly what happened to us. We would accumulate tasks and publish one or two builds per week. The effect it had on QA engineers was that, in one day, they would have a very chill day with not much to test, but suddenly the next day, they would wake up with 30+ items for them to test ASAP..
This problem can be fixed with automation by having CI set up to generate and submit builds to TestFlight every time something is merged to the development or production branches. However, this can be money and time-expensive (If you are not using the free tier of Expo, it will charge you per build). Also, we were at the beginning of the project, and the builds were generated manually by a few developers, which contributed a lot to tasks accumulating before being sent to QA.
The solution: OTA Updates + Automating the right thing
We had a few problems with our development process: A bunch of tasks that had passed code review and were ready for QA would sit for a while waiting for deployment. QAs had an inconsistent work schedule because their work would come in waves, generating and publishing new builds was a manual process that was time expensive and only a few people knew how to do it.
The first thing we did was to automate the build process, and this fixed the problem of deployments depending on a few people. However, we stumbled upon another problem: our team was working at an extraordinary velocity, and we were having a good number of deployments every day. There were some concerns about the cost of these deployments, and also, it was kind of confusing for QAs to check all the time if they were using the correct build. Another problem was that deployments would take about 30 to 40 minutes to complete, and waiting this long for a small fix is not great.
Initially, we configured the CI to run the build pipeline every time something was merged to the development branch. However, because of costs and also the amount of time required to perform each deployment, we found out that running the build process on every push was not what we wanted. We wanted this process to be more intentional, but at the same time, we could benefit from frequent deployments.
To solve this, we started implementing OTA updates, and we replaced the build workflow with the OTA update workflow. Now, when something was merged to the development branch, the CI would release an OTA update instead of creating a new build. The main advantages of using OTA updates instead of creating new builds was that: the deployment time was significantly shorter than creating a new app binary; it was cheaper than creating new builds for Android and iOS; it required just one deployment for changes to be sent to Android and iOS; and QAs didn’t have to worry about downloading new builds all the time (except for changes that require a new build, but they were less frequent).
What changed after implementing OTA updates?
The team started to do multiple OTA update deployments every day, and they were taking about 12 to 15 minutes to finish. Faster deployments allowed developers to have quick interactions with QAs to fix small bugs with ease. Before that, QAs would find a typo and send the ticket back to development, the developer would fix it, and after a few days, the ticket would be in the hands of the QA again. Now, QAs could simply message developers “Hey, your ticket is almost passing, we just need to fix this typo”, and 15 minutes later the fix was there ready for the QA to give the feature another try.
The same kind of interaction also happened with product managers and product owners when they wanted to make small adjustments to a feature before deploying it to production.
The problem of accumulating tasks before sending them to QA was completely solved, and QAs started having a regular flow of new tasks being sent to their queue every day. Also, testing most features required less work from QAs, since they just had to close and reopen the app to download the latest update. In a way, the process became very similar to testing a web application, but instead of hitting the refresh button on the browser, they had to close and reopen the app.
Problems and limitations
You will still have to create builds
Unfortunately, OTA updates have a limitation. They can’t be used to deploy changes in the native code of the mobile app. Because of that, it is necessary to create new builds every time we install a library that has native code.
Fortunately, over time, most projects tend to reach a stable version, where most libraries the app needs to work are already installed, leading to a reduced frequency of new builds. At one point during the project, we realized we were using an outdated build only because it had expired on TestFlight. Since TestFlight builds remain accessible for up to 90 days, we likely would have continued using it if it hadn’t expired.
Runtime version and update compatibility
When adding a new native library to the project, you need to change the runtime version and create a new build. The runtime version is used to guarantee compatibility between the build and OTA updates. The build will only receive updates with the same runtime version. If you forget to update the runtime version, a build without all the required native dependencies will receive the update and will crash. Always update your runtime version when installing/upgrading native dependencies, and let your team know they need to download a new build to continue receiving updates.
You will have to find a way for people to check if they are in the correct version
Sometimes, QAs were not sure if they were on the latest update or if they were using the correct app version. We solved that by adding the app version and the date and time of the latest OTA update in the app’s About screen.
Soon, it became part of the QA process, and our conversations would often start with “Hey, I’m not seeing these changes on the app, my app says version 1.3.0, and the last OTA was in… Is this correct?”. And with this info, we could check if they were using the correct version, or if the feature was simply not working.
Conclusion
Generating new builds normally takes time. OTA updates are much faster, and they facilitate more efficient interactions with QAs and stakeholders. Found a typo during a demo? Easy, just publish a fix with an update. By using OTA updates in staging, we were able to test features with ease and speed up or development process. At the end of each sprint or release cycle, we would create a production build and release it to the App Store with confidence that everything was working as expected. This was the dynamic that was introduced after we implemented OTA updates in staging.
We also used OTA updates in production to deploy small features and bug fixes, but this was not the focus of this article. Although it is great to have OTA updates in production, in my view, it was on staging that OTA updates made a huge difference. Our development process became more efficient, it reduced the pressure on QAs, gave more visibility to stakeholders to check the progress of features, and improved the communication between developers and QAs.
References
EAS build
https://docs.expo.dev/build/introduction/
EAS Update
https://docs.expo.dev/eas-update/introduction/
https://docs.expo.dev/versions/latest/sdk/updates/
How EAS Update works
https://docs.expo.dev/eas-update/how-it-works/
Runtime versions and updates
https://docs.expo.dev/eas-update/runtime-versions/
We want to work with you. Check out our Services page!
