November 14, 2023
Introduction
Our each day duties as engineers typically contain implementing new functionalities. Existing customers get to take pleasure in the advantages of those options, new customers are enticed to enroll in the app, and we get to write down extra code. At first look, it seems to be an ideal symbiotic relationship — a win-win-win state of affairs the place everyone’s pleased and nothing goes improper, proper?
But generally a shiny new function brings extra hurt than good. The cause is straightforward — software dimension. Any addition to the appliance — be it code for a brand new function, a picture useful resource for a brand new button and even assist for a brand new localization — contributes to the rise of the appliance’s dimension. You might ask, “Is this really a problem in our modern world?” We would argue that it’s.
So, what precisely is app dimension?
While the reply could seem easy, it really isn’t that easy. We can speak about app sizes in no less than 4 methods:
- Download dimension:
The dimension of the app when downloading the app from the consumer’s app retailer, i.e., the quantity of knowledge transferred over the web. To switch as little as doable over the web the app obtain dimension is compressed. - Install dimension:
The dimension of the app instantly after set up on a cell gadget, i.e., the quantity of knowledge saved on disk after an app has been put in. To have the ability to use the appliance, the app set up dimension is the content material of the appliance in uncompressed type. - Storage dimension:
The dimension of the app when it’s in use, i.e., the set up dimension plus the area taken up by varied caches (audio information, photographs, and so forth.) saved on the gadget. - Update dimension:
The dimension of the app when downloading an replace from the App Store or Play Store. This may be smaller than obtain dimension, relying on how a lot of the app has modified.
Each of those app sizes impacts customers in another way, however we are able to usually categorize their impacts underneath two domains: community associated and storage associated.
Why is app dimension so vital?
In the digital age, an increasing number of communities can embrace quick, reasonably priced cell connections, with ubiquitous Wi-Fi entry in cities and limitless fiber connections in properties, together with as much as 1TB of space for storing on smartphones and tablets. While that is true for some, our knowledge reveals {that a} substantial variety of people in rising markets use older-generation smartphones with much less space for storing — e.g., iPhone 6s with 16GB of space for storing, a portion of which is already claimed by iOS itself. Consequently, these customers are pressured to decide on which content material is most beneficial to them and may be saved on their units. Given that rising markets account for a staggering 85% of the world’s inhabitants, app dimension turns into exponentially extra vital for our customers throughout the globe. While these challenges could seem theoretical, we possess knowledge that show how actual these points are.
Google performed an experiment offering customers with an identical builds of apps, artificially inflated in dimension. The experiment revealed that for each 6MB enhance to an app’s dimension, the app’s installation-conversion charge decreased by 1%. The implications are profound, resulting in a mess of missed alternatives.
However, there exists a facet of app dimension that’s much less obvious however equally consequential.
We typically hear about world warming and the consequences automobiles, cows, and energy vegetation have on it, however the results of our apps — not simply utilizing apps however downloading updates, as effectively — aren’t often related to this situation. Every week, hundreds of thousands of customers obtain new variations of their apps. For instance, we discovered that Spotify apps (iOS and Android) in 2022 have been up to date greater than 20 billion occasions. Multiply this quantity by the obtain dimension of the apps, and also you’ll get a staggering 930+ petabytes of community site visitors. We can map this site visitors to the ability consumption for transmission. Utilizing the one-byte mannequin developed by the Shift Project, we discovered that transferring one byte by the community consumes 1.52e-10 kilowatt-hours (kWh) for Wi-Fi and eight.84e-10 kWh for cell. So even when we assume an optimistic state of affairs of 100% Wi-Fi utilization, this site visitors interprets to a minimal energy consumption of roughly 150,000,000 kWh. There is not any exact option to convert this to CO2 emission, however we are able to take EPA knowledge for 2019 and deduct that this equates to round 65,000 tonnes of CO2.
To put this determine into perspective — 65,000 tons of CO2 is roughly equal to the emission generated by 65,000 people taking a spherical journey flight between London and New York. So even when we lower the app dimension of each our apps by simply 1MB, we’d scale back our yearly carbon footprint by 560 tons of CO2 yearly. By making our apps smaller we’re not solely serving to our customers avoid wasting area on their units, but additionally serving to make our planet a bit cooler.
App-size security nets
To guard towards uncontrolled enhance in app dimension, we’ve included security nets into varied factors of our improvement course of.
Pre-merge course of:
Every time somebody initiates a pull request (PR) in our shopper repositories, an automated test swings into motion to guage how this PR impacts app dimension. These checks are a compulsory a part of CI, and the code can’t be merged till these checks are accomplished.
To carry out this test, we depend on two important builds: the bottom construct, which displays the present state of the grasp department, and the top construct, which mirrors the grasp department however contains the modifications from the PR, rebased on high of it. Once we now have these two builds prepared, we submit them for processing by Emerge Tools.
Emerge processes these builds and analyzes the variations in app dimension between them. Subsequently, we obtain a complete report with the outcomes of the test.
Once the outcomes can be found and reported again from Emerge, the report is conveniently accessible instantly throughout the pull request.
When taking a look at this knowledge, how do we all know if the advantages of the modifications we wish to merge cowl the prices of corresponding app-size enhance or not? For that objective, we’ve established a threshold of 50KB. Everything beneath that threshold may be safely merged with none further motion from the developer. The code modifications that exceed the brink are blocked and an approval is required from a devoted staff that displays and observes these instances. This staff takes a number of elements into consideration when figuring out whether or not a pull request that surpasses this threshold may be merged.
Post-merge course of:
To improve observability and permit us to maintain an in depth eye on the influence the modifications have made, we’ve arrange a devoted Slack channel the place we are able to obtain details about each pull request that will get merged and exceeds the brink. Having all these important contributions to our app dimension neatly organized in a single place makes monitoring and observing these modifications considerably simpler.
This channel isn’t solely for figuring out issues — it’s additionally a spot for celebrating successes and sharing constructive information. The identical 50KB threshold applies to PRs that efficiently scale back the app’s dimension. If a code change positively impacts app dimension and exceeds the brink ( lower than 50KB), a notification is displayed throughout the identical channel.
By implementing these app-size checks and notifications, we successfully handle our app’s dimension whereas making certain that each the event staff and the stakeholders keep a transparent view of the influence they’ve made by these modifications.
Attribution to proprietor groups
Having security nets is an effective way to forestall huge modifications from sneaking into our codebase. But most pull requests are usually smaller and don’t set off the brink alarm, so there are nonetheless challenges in the case of this sluggish “under the radar” development. Over time, these seemingly innocent additions to the codebase begin to accumulate. Given the amount of pull requests we merge each day, it’s solely pure that our app’s dimension would step by step broaden. It’s like a snowball impact — every PR, irrespective of how small, contributes its tiny piece to the puzzle. So whereas the alarms would possibly keep silent, the app dimension continues to develop.
To conquer this drawback, we’re evaluating the obtain and set up dimension of every compilation unit and useful resource file in our codebase, whereas additionally attributing it to the devoted staff that owns that module.
In distinction to PR checks, we solely want one construct for this job — particularly, the latest snapshot of the grasp department on the time of execution. We add this construct to Emerge for evaluation, and in return, because of Compilation Unit Attribution, we obtain an in depth breakdown of compilation models and the way a lot every one contributes to each the obtain and set up sizes. We then enrich this knowledge with useful resource sizes collected utilizing an in-house set of scripts.
After gathering all this knowledge, we course of it and group the information by modules. And since every module has its personal oversight staff, we are able to pinpoint who’s answerable for what.
The fantastic thing about all of it is that the information aren’t simply uncooked numbers. It’s introduced in a user-friendly interface that enables every staff to visualise and preserve tabs on their app-size contributions over time.
To make issues much more insightful, we additionally present statistics that evaluate every staff’s app-size contribution to different groups inside Spotify.
Establishing attribution and holding every staff answerable for its influence permits us to successfully keep a steadiness between rolling out new options and making certain that our app dimension stays inside affordable limits.
Working with app dimension in observe at Spotify
Why it’s more durable than it sounds
It’s simple to say that builders ought to prioritize optimizing their code for dimension, however it’s not all the time that easy.
One cause for that is that optimizing for dimension will not be a free motion. Whenever you refactor a chunk of code in order that it ends in a smaller binary, you’re very doubtless transferring that drawback to a unique space, which is most frequently both efficiency — the code is now slower because of this — or structure — the code is now more durable to learn/handle. In different phrases, optimizing for dimension is a trade-off, and the conclusion is drawn that this trade-off will not be value it.
Another cause is that generally lowering the dimensions influence of a selected piece of code merely isn’t possible. We discover that the majority pull requests that set off our alerting programs are comparatively simple to repair, however there are lots of instances the place the supply of the pointless overhead lies not within the code itself, however somewhat a particularity of how the programming language — and even the working system itself — works. While these should not unimaginable to deal with, they typically require altering how the app is structured in methods so excessive that we can’t justify doing so simply to keep away from one regression.
The third and ultimate cause is that generally you merely can’t run away from it. Any added code will lead to a bigger binary, so if you happen to’re engaged on an enormous and sophisticated function, it could merely be the case that growing the app’s set up dimension can’t be averted. To account for instances like these, we now have created one thing that we name the App Size Policy.
App Size Policy & exception Process
The coverage is, in essence, an enormous doc that explains why we care about app dimension and the programs round it, and it contains basic platform-specific recommendations on easy methods to optimize for code dimension. But most significantly, it defines an exception course of that may be adopted when builders imagine their pull requests must be allowed to significantly regress the app’s set up dimension.
In most instances, the method to be granted an exception requires proof that the proposed change may have a constructive influence on the enterprise.
Conclusion
To be certain that our instruments convey worth, we developed a set of metrics and dashboards that we constantly monitor.
| A take a look at the previous six months | ||
| 109 PRs triggered warnings |
53 of the 109 warnings have been revised and up to date to keep away from important dimension will increase |
10.5MB whole PR dimension decreases for PRs with warnings |
However, the impact of the checks extends past PRs that increase warnings or dashboards that present helpful app-size knowledge. We obtain app-size-related questions within the devoted Slack channel repeatedly, and we now have discovered that Spotify engineers like to make their PRs smaller, leading to a roughly calculated whole PR dimension lower of 20MB.
App-size development is tough to keep away from when including new options or making enhancements. But with the right steering, monitoring, and alerts, we are able to make educated choices about what’s and what isn’t value including — and when a brand new performance may end up in a win-win-win state of affairs.
Tags: engineering management, Mobile