• State Suggested Answer
  • Replies 2 replies
  • Answers 1 answer
  • Subscribers 393 subscribers
  • Views 2857 views
  • Users 0 members are here
Options
Related

System.OutOfMemoryException when looping TranslationMemoryImporter.Import()

Tristen Tan

Hi everyone,

 

I'm getting System.OutOfMemoryException when looping through the following code many times (once every 2 mins for 7 hrs)

 

public void UpdateTM(string[] tmPaths, string[] sdlxliffPaths)
{
    FileBasedTranslationMemory tm;
    TranslationMemoryImporter importer;

    foreach (string sdlxliffPath in sdlxliffPaths)
    {
        foreach (string tmPath in tmPaths)
        {
            try
            {
                tm = new FileBasedTranslationMemory(tmPath);
                importer = new TranslationMemoryImporter(tm.LanguageDirection);
                importer.ImportSettings.CheckMatchingSublanguages = true;
                importer.ImportSettings.ExistingFieldsUpdateMode = ImportSettings.FieldUpdateMode.Merge;
                importer.ImportSettings.ExistingTUsUpdateMode = ImportSettings.TUUpdateMode.Overwrite;
                importer.Import(sdlxliffPath);
                Logger.Trace("TranslatorPost.UpdateTM", " Successfully merged " + sdlxliffPath + " into " + tmPath);
            }
            catch(Exception ex)
            {
                continue;
            }
        }
    }
}

 

The error is:

07:39:45.4842 ERROR LogManager.Logger.Error : System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Xml.XmlDocument.CreateElement(String prefix, String localName, String namespaceURI)
at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)
at System.Xml.XmlLoader.ReadCurrentNode(XmlDocument doc, XmlReader reader)
at System.Xml.XmlDocument.ReadNode(XmlReader reader)
at Sdl.FileTypeSupport.Bilingual.SdlXliff.SdlXliffFeeder.GetXmlElement(XmlReader xmlReader)
at Sdl.FileTypeSupport.Bilingual.SdlXliff.SdlXliffFeeder.<ContinueScanning>b__8(ISdlXliffStreamContentHandler handler)
at System.Collections.Generic.List`1.ForEach(Action`1 action)
at Sdl.FileTypeSupport.Bilingual.SdlXliff.SdlXliffFeeder.ContinueScanning()
at Sdl.FileTypeSupport.Bilingual.SdlXliff.XliffFileReader.ContinueParsing()
at Sdl.FileTypeSupport.Bilingual.SdlXliff.XliffFileReader.ParseNext()
at Sdl.FileTypeSupport.Framework.Integration.FileExtractor.ParseNext()
at Sdl.FileTypeSupport.Framework.Integration.MultiFileConverter.ParseNext()
at Sdl.FileTypeSupport.Framework.Integration.MultiFileConverter.Parse()
at Sdl.LanguagePlatform.TranslationMemoryApi.TranslationMemoryImporter.ImportBilingualFile(String fileName)
at Sdl.LanguagePlatform.TranslationMemoryApi.TranslationMemoryImporter.Import(String fileName)
at TranslatorPost.TranslatorPost.UpdateTM(String[] tmPaths, String[] sdlxliffPaths) in D:\Visual Studio 2017 Projects\Translator\TranslatorPost\TranslatorPost.cs:line 39
Rate translation
×
Suggest better translation
×
  • 0

    Hi Tristen,

    I tried this with one TM and one sdlxliff for a start. It worked without problems. What I suspect might be an issue in your code is the order of your loops. You keep creating tm instances for each document repeatedly. I would suspect this to be comparably inefficient and it might also cause deadlocks, e.g. with your anti virus and/or with cloud backup/storage if present.

    Try swapping your loops, also maybe a GC.Collect might help a little.

    The swapping of the loop should also help in case it is just one specific TM or one specific sdlxliff causing the problem.

    Something along the line of this:

    foreach (string tmPath in tmPaths)
    {
     try
     {
        tm = new FileBasedTranslationMemory(tmPath);
        importer = new TranslationMemoryImporter(tm.LanguageDirection);
        importer.ImportSettings.CheckMatchingSublanguages = true;
        importer.ImportSettings.ExistingFieldsUpdateMode = ImportSettings.FieldUpdateMode.Merge;
        importer.ImportSettings.ExistingTUsUpdateMode = ImportSettings.TUUpdateMode.Overwrite;
        foreach (string sdlxliffPath in sdlxliffPaths)
        {
            importer.Import(sdlxliffPath);
            Logger.Trace("TranslatorPost.UpdateTM", " Successfully merged " + sdlxliffPath + " into " + tmPath);
        }
     }
     catch(Exception ex)
     {
        continue;
     }
     finally
     {
        GC.Collect();
     }
    }

     

    Hope this helps.

    Best,

    Andreas
  • 0 in reply to Andreas Galambos
    Hi Andreas,

    Thank you very much for helping to test this, and I apologise for the long delay in replying to your answer.
    I actually found that the issue is with Memory Leak when using RunAutomaticTask, as detailed in another post of mine:
    community.sdl.com/.../17849
    The workaround I currently have is to reuse the FileBasedProject instance..