Background
In QQ Reader, there used to be a noticeable stall on first launch after entering the reading page, opening the top and bottom menus, and tapping Settings. After that first interaction, the same operation would no longer show the same lag.
First Round of Analysis and Fix
After measuring with Instruments, I found that most of the time was
spent loading images through [UIImage imageNamed:]. At that
point, it became understandable why the lag only appeared the first
time.
By hooking [UIImage imageNamed:], I intercepted and
printed all image resources used by the Settings panel. Surprisingly, it
needed to load this many images:
The first idea was to load these local image resources asynchronously, but that felt like a lot of work. So I used another approach: after entering the reading page, add an observer and proactively preload these images when the main thread is idle. Once all images are loaded, remove the observer.
After this optimization, the subjective experience improved a bit. I also compared the numbers:
| iPhone X comparison | Before | After |
|---|---|---|
| First launch | 0.9481920s | 0.0424747s |
| Non-first launch | 0.0322121s | 0.0330871s |
| Non-first launch | 0.0308346s | 0.0291347s |
As the table shows, after the optimization, first-launch time became close to non-first-launch time. But this still left room to dig deeper: why did the first load have to be slower in the first place?
Although the improvement was obvious, it was not very stable. Sometimes the lag would still appear for no clear reason. Compared with shaving off more time, this instability was more worth investigating. I tested several scenarios:
- Kill the process, relaunch the app, enter the reading page, and tap Settings for the first time. The lag was not obvious.
- Send the app to the background while staying on the reading page, bring it back to the foreground, then tap Settings. The lag was reproducible.
- Rotate from portrait to landscape or from landscape to portrait. Sometimes there was lag.
For the second case, I suspected that the system cache inside
UIImage was cleared when the app entered the background. So
after returning from background to foreground, I triggered one more
preload pass on the reading page.
For the third case, I suspected that the system cache inside
UIImage was also cleared during rotation.
There was another case as well: when memory pressure is high, the
system cache inside UIImage may also be cleared.
After repeated tests involving rotation, background and foreground switching, night-mode switching, killing and restarting the app, the result became much more stable than before. This approach still had one flaw: if the user returned from the background and immediately tapped Settings, there could still be some lag because the images had not finished preloading yet. Even though the preload work was scheduled during main-thread idle time, the total image loading cost itself did not disappear. That cost still had to be paid somewhere.
Second Round of Analysis and Fix
Just when I thought the problem was solved, another teammate
suggested that simply moving the images into image.xcassets
could solve this performance issue. Out of curiosity, I moved all images
mentioned above, 27 images in total, into image.xcassets,
wrote a test method to load them all at once, and compared the result
with the original folder-based version.
The comparison was as follows:
| iPhone X comparison | Image folder | image.xcassets |
|---|---|---|
| First launch | 0.99100500s | 0.00879625s |
| Non-first launch | 0.00311238s | 0.00295617s |
| Non-first launch | 0.00290383s | 0.00289400s |
The loading speed of images inside image.xcassets was
absurdly fast. It was not even on the same order of magnitude as loading
images from a normal folder. The difference was two orders of
magnitude.
After moving all images into image.xcassets, I compared
the speed before and after the first optimization again:
| iPhone X comparison | Before | After |
|---|---|---|
| First launch | 0.0384008s | 0.0360755s |
| Non-first launch | 0.0348377s | 0.0342009s |
| Non-first launch | 0.0312605s | 0.0307469s |
At this point, the advantage of the first optimization had basically disappeared. In this situation, keeping that optimization code no longer made sense. So the result of the second round was:
Remove the first optimization code and move the images into
image.xcassets.
Yes. Extremely awkward.
Exploring Asset Catalog
The second round of optimization successfully sent me back to the
Stone Age. What puzzled me was this: why were images inside
image.xcassets loaded so incredibly fast?
Asset Catalog is Xcode’s resource management tool. Its core idea is to configure resources by device traits, including images, sprites, textures, ARKit resources, colors, data, and PDF files. This saves developers from manually configuring resource variants in code and also gives Apple better control over app package size.
In WWDC 2018: Optimizing App Assets, Apple mentioned several relevant points.
Automatic Image Packing
Before Asset Catalog classification, images were usually placed directly inside the project bundle. This approach has several drawbacks:
- Each image resource stores its own metadata and other attributes. When many resources of the same type exist, repeated metadata creates redundancy and wastes space.
- Compression happens only at the file level, so small resource files are not compressed as effectively as they could be.
- Organizing many scattered image files is cumbersome, and the APIs used to load them may differ.
- Image formats and attributes may be inconsistent. For example, some images in a group may support an alpha channel while others may not.
Asset Catalog classifies images according to their image map and stores them as atlases. It helps with both image compression and metadata storage.
- Metadata only needs to be stored once for the same kind of image resource.
- Image data can be compressed more fully. The larger the image resources are, the more obvious the optimization from Asset Catalog classification becomes.
This storage approach is somewhat similar to creating a
texture atlas. Wikipedia describes it like this:
- A texture atlas, also called a sprite sheet or an image sprite, is an image containing a collection of smaller images, usually packed together to reduce the atlas size.
This is very common in game development.
Lossy Compression
Asset Catalog supports HEIF, or
High Efficiency Image File Format, and uses it as the
default lossy compression format for Asset Catalog assets.
- It provides a higher compression ratio than existing image formats.
- It supports transparency.
- Asset Catalog can automatically convert other image types into HEIF.
Lossless Compression
Lossless compression is the default compression mode. Most assets in a project use lossless compression.
Image resources can be divided into two categories according to their color spectrum and contours. Different forms use different optimization methods during lossy compression. One category is simple image resources, which use simple color combinations and simple designs, such as many app icons. The other category is complex image resources. Lossy compression applies different optimization strategies to both forms.
Apple Deep Pixel Image Compression is a newly introduced compression format from Apple. It is flexible and chooses the best algorithm according to the color characteristics of the image.
- It adapts to the image’s color spectrum.
- It selects the best compression algorithm.
- It can improve compression size by 15-20%.
App Thinning
Each device gets only what it needs.
Conclusion 1
As we can see, App Assets can effectively reduce app package size through classification, compression, and related mechanisms. Smaller resources can indeed improve loading speed. But this still does not explain why the loading speed difference can reach two orders of magnitude.
Exploring .car Files
After the build process finishes, all resources inside Asset Catalog
are compiled into a .car file. Apple does not say much
about the structure of .car files:
If your project has an iOS 7 deployment target, Xcode compiles your asset catalogs into a runtime binary file format that improves the speed of your app.
File Structure
After looking up some references, I found that a .car
file is actually a special kind of BOM file. BOM, short for Bill of
Materials, is a file format inherited from NeXTSTEP. You can run
man 5 bom in the command line:
1 | The Mac OS X Installer uses a file system "bill of materials" to determine which files to install, remove, or upgrade |
Similar to a BOM file, a .car file contains several
block structures:
1 | CARHEADER // header information |
It also contains several tree structures:
1 | FACETKEYS |
On macOS, the private framework CoreUI.framework is
responsible for parsing .car files. During the process, it
calls C APIs from Bom.framework to parse the BOM:
1 | typedef uint32_t BOMBlockID; |
An Example
This Asset Catalog contains the following files:
- A PNG with three resolutions:
@1x,@2x, and@3x - A PDF
- A text file
- A JPG image
- A color, red with 50% transparency
After compilation, all resources inside the Asset Catalog are
compiled into Assets.car. The suffix means Compiled Asset
Catalogs.
Run this command:
1 | assetutil -I Assets.car |
1 | [ |
Take loading the PNG image as an example:
1 | UIImage *myImage = [UIImage imageNamed:@"MyImage"]; |
When the app loads MyImage from assets, it eventually
finds the binary stream shown in the image and uses
CUIThemePixelRendition to parse the data and obtain the
image information.
1 | struct CUIThemePixelRendition { |
1 | if(csiHeader->pixelFormat == 'ARGB' || csiHeader->pixelFormat == 'GA8 ' || |
This is the binary stream corresponding to the image:
The colors mean the following:
- Blue:
csiheader, fixed length of 184 bytes- Green: TLV list
- Pink: tag marker,
CELM- Yellow: version number, always set to 0
- Dark blue: compression algorithm,
RenditionCompressionType- Watery red: image data length
- Red: image data, possibly compressed
The compression algorithms include these types:
1 | enum RenditionCompressionType |
For more details, see Reverse engineering the .car file format.
Conclusion 2
- What can be confirmed here is that when loading images from assets, the process no longer performs
readI/O system calls. Instead, it directly obtains the binary stream data of the image. This seems to get us closer to the answer.- The process involves two private frameworks:
CoreUI.frameworkandBOM.framework.
Experimental Verification
Call Stack
By hooking NSCache’s setObject:forKey:cost:
method, I confirmed the call stack of
[UIImage imageNamed:]. From the stack, we can see the use
of UIAssetManager, as well as CUICatalog from
CoreUI.framework.
Priority
During implementation, I found an interesting behavior: if the normal
folder and assets both contain images with the same name but different
content, loading that name with [UIImage imageNamed:] shows
the image from assets.
If assets do not contain that image, then the image from the normal folder is displayed.
Internally, imageNamed calls a class named
UIAssetManager. Each bundle has one
UIAssetManager. It has a strong-to-strong
NSMapTable property used as a cache. Based on this, I infer
that imageNamed loads images in roughly this order:
- First, look in the cache. If it hits, return directly.
- If the cache misses, continue searching in
Assets.car. This part is handled by the privateCoreUI.framework, which decompresses, with caching, theAssets.carfile and decodes the image.- If it still cannot be found, use the bundle path and search according to rules such as
@3x,@2x,@1x,.png, and so on, until a non-Assets.carbundle image is found and loaded.
Virtual Memory and I/O
Reading an Image from Assets
1 | - (void)btnAction |
When loading an image from assets for the first time, the
.car file is opened first.
Also, .car loading is lazy. Since no images were loaded
during startup in this experiment, I did not see the .car
file being loaded at startup. It was only when the image was read for
the first time that the .car file read was triggered. As
mentioned earlier, regardless of whether an image is actually in assets,
UIAssetManager is queried once, so the .car
file open operation is triggered in the same way.
Notice the Requested Bytes in the image. No matter how
large the image being read is, this value is always 512 bytes. In the
experiment, I added a 2 MB image to assets and loaded it, and this value
was still 512 bytes.
At the same time, it also triggers virtual memory mapping. More precisely:
The image loading logic
[UIImage imageNamed:@"title_page_card_placeholder"];
triggers VM:CoreUI image file.
The image rendering logic _tempView.image = image, on
the other hand, triggers VM:IOSurface. Its
Responsible Library is CoreUI, and its
Responsible Caller is
__csiCompressImageProviderCopyIOSurfaceWithOptions.
Remember the CoreUI.framework mentioned earlier?
Remember the blue 184-byte csiheader mentioned
earlier?
Does this line up?
Judging from the naming, this is basically an in-memory copy.
Reading an Image from a Normal Folder
- First, it triggers I/O:
1 | - (void)touchesBegan:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event |
- Then the loading logic
[UIImage imageNamed:@"header"];triggersVM:CoreUI image file, which is similar to the assets case above.
- Finally, the rendering logic
_tempView.image = image;triggersVM:ImageIO_PNG_Data. ItsResponsible LibraryisImageIO, and itsResponsible CallerisImageIO_Malloc.
Conclusion 3
- The
.carfile is not loaded into memory as a whole. On first load, it uses virtual memory. When loading a specific image, if the corresponding physical page has not yet been loaded into memory, a page fault occurs. After the physical page is loaded into memory,BOM.frameworkis used to locate the corresponding binary stream. - Reading images from a normal folder involves I/O operations. When there are many images, this clearly slows things down.
- Both approaches use virtual memory. The latter also uses memory
mapping,
mmap, when calling ImageIO.
Further Verification: Update
In the sections above, I kept looking for evidence to explain why
[UIImage imageNamed:] is so surprisingly fast when loading
assets, but I did not mention
[UIImage imageWithContentsOfFile:].
In fact, I had compared it during the earlier verification process. I
used the same image from a normal folder, cold-started twice, and loaded
it separately with [UIImage imageNamed:] and
[UIImage imageWithContentsOfFile:]. The result was:
1 | Time taken for [[UIImage imageWithContentsOfFile:]] =0.0204472 s |
At first, I did not pay much attention to this result because it seemed reasonable:
- Both loading speeds are indeed on the same order of magnitude, since both require I/O.
[UIImage imageNamed:]is relatively slower because before loading the real file, it first searches through the.carcache system. Only after it fails to find the image there does it load through the path, as mentioned earlier.
Later, however, I made a more detailed statistical comparison:
There were 10 images, screen0 through
screen9, each around 1 MB. I loaded them using the
following three methods:
| Image name | Normal folder
imageNamed |
Normal folder
imageWithContentsOfFile |
.car file
imageNamed |
|---|---|---|---|
| screen0, first load | 0.1158520s | 0.04483080s | 0.00190367s |
| screen1 | 0.0387474s | 0.00345462s | 0.00038454s |
| screen2 | 0.0434894s | 0.00318442s | 0.00032900s |
| screen3 | 0.0366609s | 0.00326258s | 0.00031320s |
| screen4 | 0.0389902s | 0.00253567s | 0.00029370s |
| screen5 | 0.0437911s | 0.00245288s | 0.00036012s |
| screen6 | 0.0390221s | 0.00206237s | 0.00030758s |
| screen7 | 0.0394655s | 0.00203346s | 0.00031808s |
| screen8 | 0.0404458s | 0.00191017s | 0.00030712s |
| screen9 | 0.0417215s | 0.00195079s | 0.00031966s |
| total | 0.4788200s | 0.06915210s | 0.00616221s |
Looking at the same data from another angle:
| Statistics | Normal folder
imageNamed |
Normal folder
imageWithContentsOfFile |
.car file
imageNamed |
|---|---|---|---|
| Average | 0.040259s | 0.002424s | 0.000326s |
| Variance | 0.0023475 | 0.000543 | 2.8735e-05 |
Seeing these results made me think again:
- All three loading methods take longer for the first image. For
imageNamed, this can be explained by.carfile loading, memory mapping, and parsing. But how shouldimageWithContentsOfFilebe explained?- Apart from the first image, normal-folder
imageNamedandimageWithContentsOfFilediffer by one order of magnitude. Both involve I/O. Is the extra “one round of checking” inimageNamedenough to create an order-of-magnitude difference?- Loading from a
.carfile is still clearly the fastest. Compared with the other two, it differs by two orders of magnitude and one order of magnitude respectively.
The Loading
Process of imageWithContentsOfFile
Here I loaded two images one after another, representing first load and non-first load:
The time difference was only about two times, which still differs somewhat from the earlier measured data. I cannot explain why for now. From the stack, the difference appears around this part:
I suspect the longer first-load time may be related to necessary initialization work.
The
Process of imageNamed Loading Images from a Normal
Folder
For images in a normal folder, imageNamed first searches
inside assets. The search process, renditionWithKey, takes
the majority of the time. Of course, the image is not found there, so it
eventually calls UIAssetManager newAssetNamed:fromBundle:
to search in the bundle. This is consistent with the earlier reasoning.
What surprised me was how much time the assets lookup consumed.
From the image, we can see the three most expensive parts of
renditionWithKey:
I wonder whether Apple will optimize this method in the future.
The
Process of imageNamed Loading Images from Assets
After looking at the previous two approaches, let us return to why
imageNamed is so fast when loading images from assets:
Reasons why first load takes longer:
- The
.carfile itself needs to be loaded once, which involves I/O. UIAssetManagerneeds to be initialized.
Reasons why imageNamed loads asset resources
quickly:
- On one hand, the
.carfile itself only needs a very limited amount of I/O, while every resource read from the bundle requires a corresponding I/O operation. - On the other hand, the information for resources inside the
.carfile has already been compiled in advance, so lookup is almost direct. Loading from a normal folder requires reading the image first to obtain its parameters, then generating a rendition and rendition key, and then performing the expensivecanGetRenditionWithKeyoperation.
There are two concepts to clarify here, quoted from this article:
- What is a rendition?
A rendition is CoreUI.framework’s term for a specific variant of an
image resource, such as @1x or @2x. Each
rendition has a corresponding renditionKey, and the
renditionKey contains different attributes used to record
the parameters of the image resource.
- What are
CUIMutableStructuredThemeStoreandCUIStructuredThemeStore?
They can be understood as image sets that contain different image resources.
Conclusion 4
From the images above, we can see the following:
- If an image resource is in a
.carfile,imageNamedusesCUIStructuredThemeStore. During Asset Catalog compilation, the relevant information insideCUIStructuredThemeStorehas already been compiled into the.carfile, which is a kind of BOM file. At runtime, it can be loaded directly.
- If the image is in the bundle,
imageNamedusesCUIMutableStructuredThemeStore.UIAssetManagercannot know the file’s information in advance, so it has to load and read the file at runtime to obtain the resource details, dynamically add it toCUIMutableStructuredThemeStore, and cache it after the first load. Naturally, this process is slower.
- For images that are not in assets, most of the time is spent in
CUIMutableStructuredThemeStore’srenditionWithKeymethod. Personally, I thinkrenditionWithKeycould use further optimization, especially given Apple’s usual pursuit of performance. TheimageWithContentsOfFiletest shows that directly reading resources from the bundle is slower, but it absolutely should not be this slow. The slowdown ofimageNamedwhen reading bundle images feels, more or less, like collateral damage. In other words, while Apple improved the speed ofimageNamedfor asset resources, it also slowed downimageNamedfor bundle resources. This explains why QQ Reader became laggy from a certain version onward, even though earlier versions were smooth. It also explains why simply moving images into assets could solve the performance issue. In any case, from now on we should be cautious:imageNamedshould not be used to read image resources from the bundle. Based on the current results, that is the slowest approach.
- Whether a resource comes from assets or from the bundle,
UIAssetManagerneeds to build its own cache system. That means assets and bundle resources need a unified set of rules. Native assets complete this standardization during compilation, so at runtime all resource information is ready. Bundle resources, however, can only complete this process after their concrete information is read at runtime.
Conclusion
- If resource files have no special requirements, migrate them to assets whenever possible. This is also Apple’s recommendation.
- For large image resources that occupy a lot of memory, it is better
to put them in the bundle and load them with
imageWithContentsOfFile, so memory can be controlled more precisely. - Asset Catalog resources load at such astonishing speed, yet Apple barely mentions it. Perhaps they are just being modest.
References
WWDC 2018: Optimizing App Assets