Hi, I've gone a little deeper and have seen that tokenizer is more like the word count and it is certainly to tokenize the segment to check the similarity with existing translation units.

But the question remains the same: how can we set these flags. The function does not seem deprecated and on our server, we have 3 different sets: noFlags, DefaultTags and AllFlags. How is this possible?

Can you help? Romulus Crisan (former member)?

Kind regards

Sébastien Desautel

Can you please let me know which API you are using?

Can you please let me know which API you are using?

Hi Romulus,
It's in the translation memory API

here the link to the flag enumeration.
http://producthelp.sdl.com/SDK/TranslationMemoryApi/4.0/html/d8cc81a4-8731-9013-55b2-bf83f1f8b000.htm

Thank you

Sébastien

Hi Sébastien, 

       The TokinezerFlags are impacting the word count flags. The correct value that you need to set on the language resource template of the TM is DefaultFlags. Recognizer, tokenizer and word count flags need to be set on the template of the TM. In your case you can either create a template and use that across the TM's or create the TM save it in the server that will create a private TM template associated with the TM and set the flags. You can set the flags on the TM : ServerBasedTranslationMemory has all the flags that will update the private language resource template. 

       I hope that shared some light on your questions. 

Best regards,

Robert

Hi Robert,

thank you for your answer. Yes that's mainly what I wanted to know. I will set the tokenizer on "defaultFlags" for every TM container.

I guess the noFlags and allFlags we found for some containers are due to old TMs in older version of Studio where the property did not exist yet. Or is there any possiblity to set other Tokenizer Flags without using the API?

Kind regards

Sébastien

Hi Sébastien,

      As far as I know not really the flags are more the responsibility of the underlying components but this defaults changed over the time so it might be that. 

Best regards,

Robert