How to clear font cache filled with emoji characters?

Question

I am developing keyboard extension for iPhone. There is an emoji screen smilar to Apples own emoji keyboard that shows some 800 emoji characters in UICollectionView.

When this emoji UIScrollView is scrolled the memory usage increases and does not drop down. I am reusing cells correctly and when testing with single emoji character displayed 800 times the memory does not increase during scrolling.

Using instruments I found that there is no memory leak in my code but it seems that the emoji glyphs are cached and can take around 10-30MB of memory depending on font size (reseach shows they are actually PNGs). Keyboard extensions can use little memory before they are killed. Is there a way to clear that font cache?

---

Edit

Adding code example to reproduce the problem:

let data = Array("☺️✨✊✌️✋☝️⭐️☀️⛅️☁️⚡️☔️❄️⛄️☕️❤️️⚽️⚾️⛳️").map {String($0)}

class CollectionViewTestController: UICollectionViewController {
    override func viewDidLoad() {
        collectionView?.registerClass(Cell.self, forCellWithReuseIdentifier: cellId)
    }

    override func collectionView(collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
        return data.count
    }

    override func collectionView(collectionView: UICollectionView, cellForItemAtIndexPath indexPath: NSIndexPath) -> UICollectionViewCell {
        let cell = collectionView.dequeueReusableCellWithReuseIdentifier(cellId, forIndexPath: indexPath) as! Cell
        if cell.label.superview == nil {
            cell.label.frame = cell.contentView.bounds
            cell.contentView.addSubview(cell.label)
            cell.label.font = UIFont.systemFontOfSize(34)
        }
        cell.label.text = data[indexPath.item]
        return cell
    }

    override func numberOfSectionsInCollectionView(collectionView: UICollectionView) -> Int {
        return 1
    }
}

class Cell: UICollectionViewCell {
    private let label = UILabel()
}

After running and scrolling the UICollectionView I get memory usage graph like this:

Answer 1

I ran into the same issue and fixed it by dumping the .png from /System/Library/Fonts/Apple Color Emoji.ttf and using UIImage(contentsOfFile: String) instead of a String.

I used https://github.com/github/gemoji to extract the .png files, renamed the files with @3x suffix.

func emojiToHex(emoji: String) -> String {
    let data = emoji.dataUsingEncoding(NSUTF32LittleEndianStringEncoding)
    var unicode: UInt32 = 0
    data!.getBytes(&unicode, length:sizeof(UInt32))
    return NSString(format: "%x", unicode) as! String
}

let path = NSBundle.mainBundle().pathForResource(emojiToHex(char) + "@3x", ofType: "png")
UIImage(contentsOfFile: path!)

UIImage(contentsOfFile: path!) is properly released so the memory should stay at a low level. So far my keyboard extension hasn't crashed yet.

If the UIScrollView contains a lot of emoji, consider using UICollectionView that retains only 3 or 4 pages in cache and releases the other unseen pages.

Answer 2

I had the same issue and tried many things to release the memory, but no luck. I just changed the code based on Matthew's suggestion. It works, no more memory problem for me including iPhone 6 Plus.

The code change is minimal. Find the change in the UILabel subclass below. If you ask me the challenge is to get the emoji images. I could not figure how gemoji (https://github.com/github/gemoji) works out yet.

    //self.text = title //what it used to be
    let hex = emojiToHex(title)  // this is not the one Matthew provides. That one return strange values starting with "/" for some emojis. 
    let bundlePath = NSBundle.mainBundle().pathForResource(hex, ofType: "png")

    // if you don't happened to have the image
    if bundlePath == nil
    {
        self.text = title
        return
    }
    // if you do have the image 
    else
    {
        var image = UIImage(contentsOfFile: bundlePath!)

        //(In my case source images 64 x 64 px) showing it with scale 2 is pretty much same as showing the emoji with font size 32.
        var cgImage = image!.CGImage
        image = UIImage( CGImage : cgImage, scale : 2, orientation: UIImageOrientation.Up  )!
        let imageV = UIImageView(image : image)

        //center
        let x = (self.bounds.width - imageV.bounds.width) / 2
        let y = (self.bounds.height - imageV.bounds.height) / 2
        imageV.frame = CGRectMake( x, y, imageV.bounds.width, imageV.bounds.height)
        self.addSubview(imageV)
    }

---

The emojiToHex() method Matthew provides returns strange values starting with "/" for some emojis. The solution at the given link work with no problems so far. Convert emoji to hex value using Swift

func emojiToHex(emoji: String) -> String
{
    let uni = emoji.unicodeScalars // Unicode scalar values of the string
    let unicode = uni[uni.startIndex].value // First element as an UInt32

    return String(unicode, radix: 16, uppercase: true)
}

---------- AFTER SOME TIME----

It turned out this emojiToHex method does not work for every emoji. So I end up downloading all emojis by gemoji and map each and every emoji image file (file names are like 1.png, 2.png, etc) with the emoji itself in a dictionary object. Using the following method instead now.

func getImageFileNo(s: String) -> Int
{
        if Array(emo.keys).contains(s)
        {
             return emo[s]!
        }   
        return -1
}

Answer 3

I am guessing that you are loading the images using [UIImage imageNamed:], or something that derives from it. That will cache the images in the system cache.

You need to load them using [UIImage imageWithContentsOfFile:] instead. That will bypass the cache.

(And if that's not the problem, then you'll need to include some code in your question so that we can see what's happening.)

Answer 4

I've been around the houses on this too, and I've come to the following conclusion after numerous tests:

While the font cache does contribute to your extension's memory footprint and the total usage in the Xcode Debug Navigator and Memory Report, it isn't treated in quite the same way as the rest of your budget.

Some people cite 50 MB as the extension limit, and on Apple docs I think I've seen either 30 or 32 MB cited. We see memory warnings at various points between 30 and 40 MB, and this is too inconsistent to be happy with any particular value, but one thing that does seem to be concrete is a memory exception that occurs at 53 MB, which is logged out by Xcode as exactly that number. If I take a blank keyboard extension and populate it with even 40 MB of image views, this is one thing, but if I use 30 MB and then 20 MB of font glyph usage, I find that my keyboard isn't shut down.

From my observations, the font cache looks to get cleaned up, but not as often as you might feel necessary (especially if you're becoming nervous when that unhelpful combined memory value exceeds 30 or 32 MB).

If you budget your own memory usage at, say, 30 MB, you should be safe, provided that you don't introduce a scenario where 23 MB (i.e. 53-30) of font glyphs are all required in one swoop. This will be influenced by how dense your emoji grid is, and possibly even the font size used. It's common understanding here that if you were to scroll from one end of your emoji collection view to the other, you'll have passed through more than 23 MB of font glyphs, but if the rest of your memory footprint is reasonable (i.e. 30 MB or below), the font cache should get a chance to clean up.

In my testing, I attempted to automate bombardment of the extension with far more font glyphs, and I think I was able to beat the font cache cleanup process, resulting in a crash.

Therefore, given the use case of a UICollectionView and how fast it can be scrolled, it may be possible to crash the application if you really pushed the 30 MB memory budget and also scrolled very quickly. You might allow yourself to hit this 53 MB hard limit.

Given all of the above - with a fully fledged keyboard extension, as long as I keep to approximately 30 MB of my own (non-font-glyph) footprint I haven't encountered a crash, even when rapidly changing emoji categories and scrolling fast. I do, however, encounter system memory warnings this way, which is the thing that re-instills doubt for me.

Another problem with this approach versus using UIImage(contentsOfFile) is that it's harder to use the memory report's overall memory footprint to scrutinise your application besides what the font cache is doing. Perhaps there's a way to separate these out, but I don't know of one.

Answer 5

Many emojis are represented by sequences that contain more than one unicode scalar. Matthew's answer works well with basic emojis but it returns only first scalar from the sequences of emojis like country flags.

The code below will get full sequences and create a string that matches gemoji exported file names.

Some simple smiley emojis also have the fe0f selector. But gemoji doesn't add this selector to file names on exporting, so it should be removed.

func emojiToHex(_ emoji: String) -> String
{
    var name = ""

    for item in emoji.unicodeScalars {
        name += String(item.value, radix: 16, uppercase: false)

        if item != emoji.unicodeScalars.last {
            name += "-"
        }
    }

    name = name.replacingOccurrences(of: "-fe0f", with: "")
    return name
}

Answer 6

In my case, the plain CATextLayer helped to reduce the memory usage of my app. When I used the UILabel to render Emojis the keyboard extension memory was increasing from ~16MB to ~76MB. After the replacement of the UILabel with the CATextLayer, the keyboard extension memory increasing from ~16MB to only ~26MB.

Previous UICollectionViewCell subclass setup:

- (instancetype)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];

    // somewhere in the header file 
    // @property (nonatomic, strong, readonly) UILabel *textLabel;
    _textLabel = [UILabel new];
    self.textLabel.font = [UIFont fontWithName:@"HelveticaNeue" size:28];
    self.textLabel.textAlignment = NSTextAlignmentCenter;
    [self addSubview:self.textLabel];
    // some auto layout logic using Masonry
    [self.textLabel mas_makeConstraints:^(MASConstraintMaker *make) {
        make.size.equalTo(self);
        make.center.equalTo(self);
    }];

    return self;
}

My UICollectionViewCell subclass setup with the CATextLayer:

- (instancetype)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];

    // somewhere in the header file 
    // @property (nonatomic, strong, readonly) CATextLayer *textLayer;
    _textLayer = [CATextLayer new];
    self.textLayer.frame = CGRectMake(0, 0, 33, 33);
    self.textLayer.font = CFBridgingRetain([UIFont fontWithName:@"HelveticaNeue" size:28].fontName);
    self.textLayer.fontSize = 28;
    self.textLayer.alignmentMode = kCAAlignmentCenter;
    [self.layer addSublayer:self.textLayer];

    return self;
}

Update

Sorry guys forgot to add the self.textLayer.contentsScale = [[UIScreen mainScreen] scale]; to get clear text. That unfortunately increased usage of memory from ~16MB to ~44MB, but still better than the UILabel solution.

Final UICollectionViewCell subclass setup with the CATextLayer:

- (instancetype)initWithFrame:(CGRect)frame {
    self = [super initWithFrame:frame];

    [self.layer setRasterizationScale:[[UIScreen mainScreen] scale]];

    // somewhere in the header file 
    // @property (nonatomic, strong, readonly) CATextLayer *textLayer;
    _textLayer = [CATextLayer new];
    self.textLayer.frame = CGRectMake(0, 0, 33, 33);
    self.textLayer.font = CFBridgingRetain([UIFont fontWithName:@"HelveticaNeue" size:28].fontName);
    self.textLayer.fontSize = 28;
    self.textLayer.alignmentMode = kCAAlignmentCenter;
    NSDictionary *newActions = @{
        @"onOrderIn": [NSNull null],
        @"onOrderOut": [NSNull null],
        @"sublayers": [NSNull null],
        @"contents": [NSNull null],
        @"bounds": [NSNull null]
    };
    self.textLayer.actions = newActions;
    [self.layer addSublayer:self.textLayer];

    [self.layer setShouldRasterize:YES];

    return self;
}