Skip to content

Normalize text for downstream systems

Fold Unicode punctuation to ASCII and filter text by character class, triggered when caller prepares text for a fixed-format file, a legacy system, or display*.

Field Value
Operation ID OP-032
Trigger Caller prepares text for a fixed-format file, a legacy system, or display*
Inputs string
Outputs string
Primary components StringExtensions.ReplaceUnicodeCharactersWithAscii`; `StringUtilities.Digits` / `Letters` / `LettersOrDigits` / `Punctuation` / `Symbols` / `ControlCharacters` / `RemoveControlCharacters` / `RemoveWhiteSpaces
Entry point Common/SAIF.Common/Extensions/StringExtensions.cs

Steps

  1. ReplaceUnicodeCharactersWithAscii substitutes string.Empty for a null input (via GenericExtensions.IsNull) so it never throws. (BR-074)
  2. Fold 4 curly/angled single-quote code points to ', 6 double-quote code points to ", 6 bullet code points to *, 8 dash/hyphen code points to -, and the NUL character to a space, in one chained Replace expression. (BR-074)
  3. Each StringUtilities filter null-guards the input (returning ""), then walks the character array retaining only characters matching the corresponding char.Is* predicate (IsDigit, IsLetter, IsLetterOrDigit, IsPunctuation, IsSymbol, IsControl). (BR-075)
  4. RemoveControlCharacters and RemoveWhiteSpaces invert the predicate, retaining everything that is not a control character / not whitespace. (BR-075)

Process flow

The following diagram visualizes this operation's control flow.

flowchart TD
    Start(["Caller normalizes text for<br>a downstream system"]) --> Null{"ReplaceUnicodeCharactersWithAscii:<br>input is null?"}
    Null -->|Yes| Empty(["Return string.Empty"])
    Null -->|No| Fold["Fold quotes, bullets, dashes,<br>and NUL to ASCII equivalents"]
    Fold --> Filter["StringUtilities filter: guard input,<br>retain chars matching char.Is* predicate"]
    Filter --> Invert["Remove variants invert predicate:<br>retain non-matching characters"]
    Invert --> End(["Normalized string returned"])

Note: Statements marked with * were inferred from code structure, naming conventions, or contextual evidence where direct confirmation was not available in source artifacts.

Sources

  • Common/SAIF.Common/Extensions/StringExtensions.cs
  • Common/SAIF.Common/Utility/StringUtilities.cs