Avalonia controls localization

[maxkatz6]

Upd: closed with #13773

Currently, our controls hardcode English localization. Which makes it pretty hard to localize applications in another way.
As this task isn't trivial, I want to start at least some discussion about it, so we can initiate a solution within 11.0.

How to use and replace localized strings (most important)

1. ResourceManager. I would not, it pretty much locks users to the single approach, somewhat out-of-date.
2. IStringLocalizer - requires dependency on Microsoft.Extensions.Localization.
3. Create an interface like IStringLocalizer and provide a static property that can be replaced:
   3.1. DevExpress does it by creating a localizer class per each control. They also have ready-to-use helpers like CreateResXLocalizer().
   3.2. Telerik by default users RESX, but also allows to replace LocalizationManager
   3.3. More examples?

Dynamic language change

CultureInfo class doesn't have any events when UI thread culture was changed.
So, we have to invalidate controls when some specific Avalonia event was changed.
Some platforms also have system language changed events. Not sure though if it should be handled by Avalonia or by the developer's app.

How to store localized strings per culture

1. RESX with satellite assemblies. Pain with code generation and conflicts (ancient generator at this point). Adds problems with AOT.
2. Same RESX but source-generated constants. It might be the best solution if we want to have more than localization in the repository.
3. Keep private constants only for EN. Probably the best solution for now. While still keeping freedom of the implementation in the future without downsides like satellite assemblies.

My API proposal

Avalonia side:

public interface IAvaloniaLocalizer
{
 string this[string name] { get; }
 string this[string name, params object[] arguments] { get; }
}

public class TimePickerLocalizer : IAvaloniaLocalizer
{
 public static IAvaloniaLocalizer Active { get; set; }
 public static event EventHandler ActiveChanged;

 public const string HourCaptionId = nameof(HourCaptionId);

 private const string HourCaptionEnValue = "hour";
}

public class TimePicker
{
 protected override void OnAttachedToVisualTree()
 {
  SetSelectedTimeText();
  TimePickerLocalizer.ActiveChanged += (_, _) => SetSelectedTimeText();
 } 

 private void SetSelectedTimeText()
 {
  _hourText.Text = TimePickerLocalizer.Active[TimePickerLocalizer.HourCaptionId];
 }
}

Users code side:

public class ResxAvaloniaLocalizer : IAvaloniaLocalizer
{
 public ResxAvaloniaLocalizer(CultureInfo) {}
 public string this[string name] => ResourceManager.GetString(name, CultureInfo);
 public string this[string name, params object[] arguments] => string.Format(ResourceManager.GetString(name, CultureInfo), arguments);
}

TimePickerLocalizer.Instance = new ResxAvaloniaLocalizer(CultureInfo.CurrentUICulture);
ApplicationViewModel.UserChangedLanguage += (_, _) => TimePickerLocalizer.Instance = new ResxAvaloniaLocalizer(newCulture);

Questions to this implementation:

1. It's not a solution for a general localization of the Avalonia applications, but only for the built-in controls. And probably a template solution for the third-party controls.
2. Build-in localizations are limited to EN only, but potentially can be extended without breaking API (either satellite assemblies or source generated...)
3. There is no way to override localization per control. Do we need it? If yes, API shouldn't be changed. xml:lang?
4. IStringLocalizer returns LocalizedString with additional information, but I don't think we would need it in Avalonia code base.
5. There is no GetAllStrings alternative, but I don't see any reasons to have it in the Avalonia.
6. Should language depended assets should be supported by the AssetLoader?

Alternative solutions that also might be considered

1. Application.Current.Resources ?

[timunie]

I've my entire App localized via DynamicResources. Just like FluentTheme I can switch the lang at runtime with no effort. As the resources can be overridden, I can also have different language in a sub control. Moreover this approach would not tie us to strings only, as we can also localize images via DynRes.

See: https://github.com/timunie/Tims.Avalonia.Samples/tree/main/src%2FLocalizationSample

I've an Excel File that can create the .xaml files for me. This can also be an avalonia App we can share for anyone to manage their languages.

That said, I'm completely open to other ideas as well.

[jp2masa]

This is a complex topic, but for now my main concerns are the following:

• I personally would like to see IStringLocalizer being used, although it's not part of the BCL. It's really sad to see all frameworks defining the same interfaces all the time, and then having to write adapters to make it all work. I understand why they do it, but in this particular case it would be even worse because Microsoft.Extensions.Localization is lightweight (and if you only care about the abstractions, without resx localizer, there's Microsoft.Extensions.Localization.Abstractions). LocalizedString is in fact too complex for what Avalonia needs, may be a problem.

• There's a conceptual mistake when defining TimePickerLocalizer, as it's implicitly dependent on the template, so this doesn't sound good:

  TimePickerLocalizer.ActiveChanged += (_, _) => SetSelectedTimeText();

  TimePicker shouldn't depend on the template, so this localizer should probably be a property.

  What I will say now is completely out of scope, but honestly I think it's a problem with WPF and Avalonia to have a control template as a property, as the template is conceptually much more that just a Func<IControl>, it defines what styles are compatible/expected, what (localized) text is compatible/expected,...

• This doesn't sound good either:

  public static IAvaloniaLocalizer Active { get; set; }
  public static event EventHandler ActiveChanged;

  • Globals should be avoided
  • There may be apps which want to have windows or controls with different locales

[robloo]

This has been discussed a few places now but at a high level I think localization should be done as follows:

1. Use .resx files for all the raw strings (technically this can serialize more than text but we'll just do text for now)
   • Make sure to follow the Strings directory structure with subfolders for each culture.
   • This is pretty much a standard now
2. Use a code generator to convert the .resx files into string constants as you mentioned above. This is what is used in source code directly. I don't really mind what interface you put the lookup behind.
   • Something like this: https://github.com/VocaDB/ResXFileCodeGenerator. Might want to make one in Avalonia directly.
   • This has been done since the WinForms days for the most part
3. Localization in XAML needs to be supported as well. I've personally decided after a lot of discussion to go with a XAML markup extension to do this. That will then access the same string constants generated from the .resx files through whatever interface Avalonia provides.
   • https://devblogs.microsoft.com/ifdef-windows/use-a-custom-resource-markup-extension-to-succeed-at-ui-string-globalization/
   • Avalonia localization #3061
   • Avalonia localization #7131

[maxkatz6]

That's how to localize the XAML application.
But in this discussion, I was thinking about specifically Avalonia strings localization. Something that can be redefined and replaced by the developer. As develop might want to change localization or add more languages.

[robloo]

I realize that, I was thinking we should use the same technique for Avalonia itself. While Microsoft doesn't actually make their internal localization infrastructure available in frameworks we could. So I was thinking it could be similar for Apps and Avalonia itself. I really see the value in a standardized built-in system everyone can use (UWP was nice this way). So having full support for localization in Avalonia for app developers and the framework itself seems like a win-win -- and the tooling could be the same.

I think adding a generic IResourceLocator/ResxAvaloniaLocalizer/IStringLocalizer that accesses the generated code from the .resx files is also useful. Then devs could, as you said, swap-out the default implementation. However, a default implementation should be provided as mentioned in points 1 and 2 above. I also wouldn't really make this control-specific.

[kant2002]

1. I would like that RESX would not appears in the Avalonia. Right now Avalonia plans support aggressive AOT scenarios and relying on REsX would block this goal for foreseeable future.
2. would be great if some internal interface would be used, so external forces in localization would not affect Avalonia.
3. If IStringLocalizer would be used, so be it.

[rstm-sf]

most of .net

There are difficulties for F# :) fsharp/fslang-suggestions#802

[robloo]

Yes, so ResX files are still the source -- but aren't used directly. Those files are then passed through a code generator to produce C# or F#. But the original strings are still managed in .ResX files and can use all the tooling (both online and VS studio extensions, etc.) that provides.

I specifically said to pass the ResX files through a code generator from the beginning.

[kant2002]

I would say, I come from position what I do not want to see in Avalonia, and did not think through options how it should be implemented.

@maxkatz6 proposals seems fit very well with my limitations. I personally not against using RESX, if that can be swapped to reflection-free variant.

Where do you propose even storing source translation strings then?

Personally I have preference for XLIFF as source files to drive localization efforts. Also I would say for Avalonia MO files seems to be a better fit for storing data on disk/external storage. RESX tooling on Linux/Mac probably limiting, but XLIFF/PO are much better represented in cross-platform tools. And please no PO files by default :)

Certainly you aren't saying we should have source code files for each culture and manually maintain the source code?

I'm spend my craziness ticket on reflection-free Avalonia wish, so cannot ask for requests like that 😄

Those files are then passed through a code generator to produce C# or F#.

I did not see how reflection-free Avalonia can take off with ResourceManager and Assembly.GetManifestResourceStream. If you propose difference source generator, I do not understand where data would be stored if not in the assembly, and how code gen would switch culture. That's seems to be require new API (which maybe fine)

But the original strings are still managed in .ResX files and can use all the tooling (both online and VS studio extensions, etc.) that provides.

If ResX files would be used for source and translated resources what would be stored on disk?

[maxkatz6]

If you propose difference source generator, I do not understand where data would be stored if not in the assembly

Proposed VocaDB source generator (which I personally use as well), uses RESX files as a place to define all the string resources. And it generates C# constants for each language. So, all supported cultures are hardcoded in the assembly this way. Which is a good enough way for the applications (most of them), but not for the framework.
Though, it definitely can be used as internal implementation, while whole mechanism still can be replaced with another interface implementation.

[robloo]

Though, it definitely can be used as internal implementation, while whole mechanism still can be replaced with another interface implementation.

Yes, keep an IStringLocalizer interface (I wouldn't re-invent the wheel here as mentioned in @jp2masa comment) but behind the scenes use .RESX files and generate the code. I would also pull into the framework a source generator like VocaDB and also provide a localization markup extensions for XAML. Apps can use them if they want but it wouldn't be required. It would go a long way to standardize localization in Avalonia apps as well as provide a default implementation used by the framework itself.