使用NSXMLParser将XML解析为核心数据存储时的内存积累

I have a problem with an app that takes an XML feed, parses it, and stores the results into Core Data.

我对一个应用程序有一个问题，它接受XML提要，解析它，并将结果存储到核心数据中。

The problem only occurs on the very first run of the app when there is nothing in the store and the whole feed is parsed and stored. The problem is simply that memory allocations build up and up until, on 50% of attempts it crashes the app, usually at around 10Mb. The objects allocated seem to be CFData(store) objects and I can't seem to find any way to force a release of them. If you can get it to run once and successfully complete the parsing and save to core data then every subsequent launch is fine, memory usage never exceeds 2.5Mb

问题只出现在应用程序的第一次运行时，商店中什么都没有，整个提要被解析和存储。问题很简单，内存分配不断增加，直到50%的尝试都失败了，通常是10Mb左右。分配的对象似乎是CFData(store)对象，我似乎找不到任何方法来强制释放它们。如果您可以让它运行一次并成功完成解析并保存到核心数据，那么每次后续的启动都没问题，内存使用不会超过2.5Mb

Here's the general approach I have before we get into code: Get the feed into an NSData object. Use NSFileManager to store it as a file. Create a URL from the file path and give it to the parseXMLFile: method. Delegate is self. On reaching parser:didStartElement:namespaceURI:qualifiedName:attributes: I create a dictionary to catch data from tags I need. The parser:foundCharacters: method saves the contents of the tag to a string The parser:didEndElement:... method then determines if the tag is something we need. If so it adds it to the dictionary if not it ignores it. Once it reaches the end of an item it calls a method to add it to the core data store.

在进入代码之前，我有一个通用的方法:将提要放到NSData对象中。使用NSFileManager将其存储为文件。从文件路径创建一个URL并将其交给parseXMLFile:方法。委托是自我。到达解析器:didStartElement:namespaceURI:qualifiedName:attributes:我创建了一个字典来从我需要的标记中捕获数据。方法将标记的内容保存到字符串解析器:didEndElement:…方法然后确定标记是否是我们需要的东西。如果是这样，它会将它添加到字典中，如果不是，它会忽略它。一旦它到达项的末尾，它就调用一个方法将其添加到核心数据存储中。

From looking at sample code and other peoples postings here it seems there's nothing in the general approach thats wrong.

通过查看示例代码和其他人在这里发布的文章，似乎在一般的方法中没有什么是错误的。

The code is below, it comes from a larger context of a view controller but I omitted anything not directly related.

代码在下面，它来自一个视图控制器的大上下文，但是我省略了任何不直接相关的内容。

In terms of things I have tried: The feed is XML, no option to convert to JSON, sorry. It's not the images being found and stored in the shared documents area.

至于我尝试过的东西:提要是XML，不能转换为JSON，抱歉。不是在共享文档区域中找到并存储的映像。

Clues as to what it might be: This is the entry I get from Instruments on the largest most numerous things allocated (256k per item):

关于它可能是什么的提示:这是我从分配的最大数量的设备上得到的条目(每个条目256k):

Object Address Category Creation Time Live Size Responsible Library Responsible Caller

对象地址类别创建时间活尺寸负责库负责调用者

1 0xd710000 CFData (store) 16024774912 • 262144 CFNetwork URLConnectionClient::clientDidReceiveData(_CFData const*, URLConnectionClient::ClientConnectionEventQueue*)

1 0xd710000 CFData (store) 16024774912•262144 CFNetwork URLConnectionClient::clientDidReceiveData const*， URLConnectionClient:::ClientConnectionEventQueue*)

And here's the code, edited for anonymity of the content ;)

这里是代码，内容的匿名编辑;

-(void)parserDidStartDocument:(NSXMLParser *)feedParser { }

-(void)parserDidEndDocument:(NSXMLParser *)feedParser { }

-(void)parser:(NSXMLParser *)feedParser didStartElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qualifiedName attributes:(NSDictionary *)attributeDict
{
    eachElement = elementName;
    if ( [eachElement isEqualToString:@"item" ] )
    {
        //create a dictionary to store each item from the XML feed
        self.currentItem = [[[NSMutableDictionary alloc] init] autorelease];
    }
}


-(void)parser:(NSXMLParser *)feedParser foundCharacters:(NSString *)feedString 
{
    //Make sure that tag content doesn't line break unnecessarily
    if (self.currentString == nil) 
    {
        self.currentString = [[[NSMutableString alloc] init]autorelease];
    }
    [self.currentString appendString:feedString];

}

-(void)parser:(NSXMLParser *)feedParser didEndElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName 
{
    eachElement = elementName;
    NSString *tempString = [self.currentString stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];

if ( [eachElement isEqualToString:@"title"] ) 
    {
        //skip the intro title
        if (![tempString isEqualToString:@"Andy Panda UK"])
        {
            //add item to di citonary output to console
            [self.currentItem setValue:tempString forKey:eachElement];
        }
    }
    if ( [eachElement isEqualToString:@"link"] ) 
    {
        //skip the intro link
        if (![tempString isEqualToString:@"http://andypanda.co.uk/comic"])
        {
            //add item to dicitonary output to console
            [self.currentItem setValue:tempString forKey:eachElement];
        }
    }
    if ( [eachElement isEqualToString:@"pubDate"] ) 
    {
        //add item to dicitonary output to console
        [self.currentItem setValue:tempString forKey:eachElement];
    }
    if ( [eachElement isEqualToString:@"description"] ) 
    {
        if ([tempString length] > 150)
        {
            //trims the string to just give us the link to the image file
            NSRange firstPart = [tempString rangeOfString:@"src"];
            NSString *firstString = [tempString substringFromIndex:firstPart.location+5];
            NSString *secondString;
            NSString *separatorString = @"\"";
            NSScanner *aScanner = [NSScanner scannerWithString:firstString];
            [aScanner scanUpToString:separatorString intoString:&secondString];

            //trims the string further to give us just the credits
            NSRange secondPart = [firstString rangeOfString:@"title="];
            NSString *thirdString = [firstString substringFromIndex:secondPart.location+7];
            thirdString = [thirdString substringToIndex:[thirdString length] - 12];
            NSString *fourthString= [secondString substringFromIndex:35];

            //add the items to the dictionary and output to console
            [self.currentItem setValue:secondString forKey:@"imageURL"];
            [self.currentItem setValue:thirdString forKey:@"credits"];
            [self.currentItem setValue:fourthString forKey:@"imagePreviewURL"];
            //safety sake set unneeded objects to nil before scope rules kick in to release them
            firstString = nil;
            secondString = nil;
            thirdString = nil;
        }
        tempString = nil;
    }


    //close the feed and release all the little objects! Fly be free!
    if ( [eachElement isEqualToString:@"item" ] )
    {
        //get the date sorted
        NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
        formatter.dateFormat = @"EEE, dd MMM yyyy HH:mm:ss ZZZZ";
        NSDate *pubDate = [formatter dateFromString:[currentItem valueForKey:@"pubDate"]];
        [formatter release];
        NSArray *fetchedArray = [[NSArray alloc] initWithArray:[fetchedResultsController fetchedObjects]];
        //build the string to make the image
        NSString *previewURL = [@"http://andypanda.co.uk/comic/comics-rss/" stringByAppendingString:[self.currentItem valueForKey:@"imagePreviewURL"]];

        if ([fetchedArray count] == 0)
        {
            [self sendToCoreDataStoreWithDate:pubDate andPreview:previewURL];
        } 
        else 
        {
            int i, matches = 0;
            for (i = 0; i < [fetchedArray count]; i++)
            {
                if ([[self.currentItem valueForKey:@"title"] isEqualToString:[[fetchedArray objectAtIndex:i] valueForKey:@"stripTitle"]])
                {
                    matches++;
                }
            }

            if (matches == 0)
            {
                [self sendToCoreDataStoreWithDate:pubDate andPreview:previewURL];
            }
        }
        previewURL = nil;
        [previewURL release];
        [fetchedArray release];
    }
    self.currentString = nil;
}

- (void)sendToCoreDataStoreWithDate:(NSDate*)pubDate andPreview:(NSString*)previewURL {
    NSManagedObjectContext *context = [fetchedResultsController managedObjectContext];
    NSEntityDescription *entity = [[fetchedResultsController fetchRequest] entity];
    newManagedObject = [NSEntityDescription insertNewObjectForEntityForName:[entity name] inManagedObjectContext:context];
    [newManagedObject setValue:[NSDate date] forKey:@"timeStamp"];
    [newManagedObject setValue:[[currentItem valueForKey:@"title"] description] forKey:@"stripTitle"];
    [newManagedObject setValue:[[currentItem valueForKey:@"credits"] description] forKey:@"stripCredits"];
    [newManagedObject setValue:[[currentItem valueForKey:@"imageURL"] description] forKey:@"stripImgURL"];
    [newManagedObject setValue:[[currentItem valueForKey:@"link"] description] forKey:@"stripURL"];
    [newManagedObject setValue:pubDate forKey:@"stripPubDate"];
    [newManagedObject setValue:@"NO" forKey:@"stripBookmark"];
    //**THE NEW SYSTEM**
    NSString *destinationPath = [(AndyPadAppDelegate *)[[UIApplication sharedApplication] delegate] applicationDocumentsDirectory];
    NSString *guidPreview = [[NSProcessInfo processInfo] globallyUniqueString];
    NSString *guidImage = [[NSProcessInfo processInfo] globallyUniqueString];
    NSString *dpPreview = [destinationPath stringByAppendingPathComponent:guidPreview];
    NSString *dpImage = [destinationPath stringByAppendingPathComponent:guidImage];
    NSData *previewD = [NSData dataWithContentsOfURL:[NSURL URLWithString:previewURL]];
    NSData *imageD = [NSData dataWithContentsOfURL:[NSURL URLWithString:[currentItem valueForKey:@"imageURL"]]];
    //NSError *error = nil;
    [[NSFileManager defaultManager] createFileAtPath:dpPreview contents:previewD  attributes:nil];
    [[NSFileManager defaultManager] createFileAtPath:dpImage contents:imageD attributes:nil];
    //TODO: BETTER ERROR HANDLING WHEN COMPLETED APP
    [newManagedObject setValue:dpPreview forKey:@"stripLocalPreviewPath"];
    [newManagedObject setValue:dpImage forKey:@"stripLocalImagePath"];  
    [newManagedObject release];
    before++;
    [self.currentItem removeAllObjects];
    self.currentItem = nil;
    [self.currentItem release];
    [xmlParserPool drain];
    self.xmlParserPool = [[NSAutoreleasePool alloc] init];

}

[EDIT] @Rog I tried playing with NSOperation but no luck so far, the same me,ory build up with the same items, all about 256k each. Here's the code:

[编辑]@Rog我尝试过使用NSOperation，但到目前为止没有运气，同样的我，ory用同样的项目建立起来，每个都大约256k。这是代码:

- (void)sendToCoreDataStoreWithDate:(NSDate*)pubDate andPreview:(NSString*)previewURL
{
    NSManagedObjectContext *context = [fetchedResultsController managedObjectContext];
    NSEntityDescription *entity = [[fetchedResultsController fetchRequest] entity];

    newManagedObject = [NSEntityDescription insertNewObjectForEntityForName:[entity name] inManagedObjectContext:context];


    [newManagedObject setValue:[NSDate date] forKey:@"timeStamp"];
    [newManagedObject setValue:[[currentItem valueForKey:@"title"] description] forKey:@"stripTitle"];
    [newManagedObject setValue:[[currentItem valueForKey:@"credits"] description] forKey:@"stripCredits"];
    [newManagedObject setValue:[[currentItem valueForKey:@"imageURL"] description] forKey:@"stripImgURL"];
    [newManagedObject setValue:[[currentItem valueForKey:@"link"] description] forKey:@"stripURL"];
    [newManagedObject setValue:pubDate forKey:@"stripPubDate"];
    [newManagedObject setValue:@"NO" forKey:@"stripBookmark"];

NSString *destinationPath = [(AndyPadAppDelegate *)[[UIApplication sharedApplication] delegate] applicationDocumentsDirectory];
NSString *guidPreview = [[NSProcessInfo processInfo] globallyUniqueString];
NSString *guidImage = [[NSProcessInfo processInfo] globallyUniqueString];
NSString *dpPreview = [destinationPath stringByAppendingPathComponent:guidPreview];
NSString *dpImage = [destinationPath stringByAppendingPathComponent:guidImage];

//Create an array and send the contents off to be dispatched to an operation queue
NSArray *previewArray = [NSArray arrayWithObjects:dpPreview, previewURL, nil];
NSOperation *prevOp = [self taskWithData:previewArray];
[prevOp start];

//Create an array and send the contents off to be dispatched to an operation queue
NSArray *imageArray = [NSArray arrayWithObjects:dpImage, [currentItem valueForKey:@"imageURL"], nil];
NSOperation *imagOp = [self taskWithData:imageArray];
[imagOp start];

[newManagedObject setValue:dpPreview forKey:@"stripLocalPreviewPath"];
[newManagedObject setValue:dpImage forKey:@"stripLocalImagePath"];  
//[newManagedObject release]; **recommended by * answer
before++;
[self.currentItem removeAllObjects];
self.currentItem = nil;
[self.currentItem release];
[xmlParserPool drain];
self.xmlParserPool = [[NSAutoreleasePool alloc] init];


}

- (NSOperation*)taskWithData:(id)data
{
    NSInvocationOperation* theOp = [[[NSInvocationOperation alloc] initWithTarget:self
                                                                         selector:@selector(threadedImageDownload:) 
                                                                           object:data] autorelease];
    return theOp;
}

- (void)threadedImageDownload:(id)data 
{
    NSURL *urlFromArray1 = [NSURL URLWithString:[data objectAtIndex:1]];
    NSString *stringFromArray0 = [NSString stringWithString:[data objectAtIndex:0]];
    NSData *imageFile = [NSData dataWithContentsOfURL:urlFromArray1];
    [[NSFileManager defaultManager] createFileAtPath:stringFromArray0 
                                            contents:imageFile 
                                          attributes:nil];
    //-=====0jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjmn **OLEG**
}

3 个解决方案

#1

Take a look at AQXMLParser. There is a example app that shows how to integrate it with Core Data on a background thread. It's a streaming parser so memory usage is minimal.

看看AQXMLParser。有一个示例应用程序展示了如何将它与后台线程上的核心数据集成在一起。它是一个流解析器，所以内存使用是最少的。

Set up a second MOC and fetched results controller for the UI and refetch when the parsing is complete to get the new data. (one option is to register for change notifications when the background MOC saves to merge between contexts)

当解析完成后，为UI设置第二个MOC和fetchresultscontroller，以获取新数据。(一种选择是在后台MOC保存上下文的时候注册更改通知)

For images, there are a number of third party categories on UIImageVew that support asynchronous downloads and placeholder images. So you would parse the image URL, store it in Core Data, then set the URL on the image view when that item is viewed. This way the user doesn't need to wait for all the images to be downloaded.

对于图像，UIImageVew上有许多第三方类别支持异步下载和占位符图像。因此，您将解析图像URL，将其存储在Core Data中，然后在查看该项时设置图像视图上的URL。这样用户就不需要等待所有的图片被下载。

#2

You can disable the caching :

您可以禁用缓存:

NSURLCache *sharedCache = [[NSURLCache alloc] initWithMemoryCapacity:0 diskCapacity:0 diskPath:nil];
[NSURLCache setSharedURLCache:sharedCache];
[sharedCache release];

Or clear it :

或清除:

[[NSURLCache sharedURLCache] removeAllCachedResponses];

This should fix your problem.

这应该能解决你的问题。

#3

Your problem is here:

你的问题是:

[newManagedObject release];

(newManagedObject释放);

Get rid of it, and the crashes will be gone (you don't release NSManagedObjects, Coredata will handle that for you).

去掉它，崩溃就会消失(您不会释放NSManagedObjects, Coredata会为您处理)。

Cheers,

欢呼,

Rog

罗格

#1