Add character encoding for the official multilingual release (resolves …

…rrooij#11)

docs/language support.md

@@ -1,6 +1,16 @@
 # Language Support
 
-- Original Japanese Release
+Original Release (Super Famicom)
+
+- Original Japanese Release from 1995
+
+Collection of Mana / Seiken Densetsu Collection (Nintendo Switch)
+
+- Japanese Release from 2017
+- International Release from 2019 (English, French, German, Spanish)
+
+Fan Translations
+
 - English Translation V1.01 by Neill Corlett, SoM2Freak
 - French Translation RC1 by Terminus Traduction (Copernic)
 - German Translation V1.00 RC3 to V3.0 by G-Trans (Special-Man, LavosSpawn)
@@ -21,7 +31,16 @@ Please refer to the source code for the actual regular expressions.
     | ---| --- |
     | **Japanese** | Seiken Densetsu 3 (J) <br/> Seiken Densetsu 3 (Japan) |
 
-2. Patch name (English, French, German, Italian, Spanish, incl. Japanese)
+2. Switch release (Japanese, Multilingual)
+
+    | | |
+    | ---| --- |
+    | **Japanese** | Seiken Densetsu 3 (J) (Seiken Densetsu Collection) <br/> Seiken Densetsu 3 (Japan) (Seiken Densetsu Collection) |
+    | **Multilingual** | Trials of Mana (W) <br/> Trials of Mana (World) <br/> Trials of Mana (W) (Collection of Mana) <br/> Trials of Mana (World) (Collection of Mana) |
+
+    Looking for “Trials of Mana”, “Collection of Mana” and “Seiken Densetsu Collection”.
+
+3. Patch name (English, French, German, Italian, Spanish, incl. Japanese)
 
     | | |
     | ---| --- |
@@ -35,7 +54,7 @@ Please refer to the source code for the actual regular expressions.
 
     Only Italian differs from this pattern slightly.
 
-3. Translator / translation team (English, French, German, Italian, Spanish)
+4. Translator / translation team (English, French, German, Italian, Spanish)
 
     | | |
     | ---| --- |
@@ -45,7 +64,7 @@ Please refer to the source code for the actual regular expressions.
     | **Italian** | Mumble Translations, Clomax, Ombra, Chester |
     | **Spanish** | Magno, Vegetal Gibber |
 
-4. Language / language code (English, French, German, Italian, Spanish, incl. Japanese)
+5. Language / language code (English, French, German, Italian, Spanish, incl. Japanese)
 
     | | |
     | ---| --- |
@@ -58,12 +77,12 @@ Please refer to the source code for the actual regular expressions.
 
     Only two and three letter language codes are taken into account here, no single letter codes.
 
-5. Fallback to English
+6. Fallback to English
 
 ### Remarks
 
 - File names are treated case-insensitive.
-- There is also a pre-patched ROM circulating named `Seiken Densetsu 3 (Japan) [En by LNF+Neill Corlett+SoM2Freak v1.01]`. This is covered by rule 3 and 4.
+- There is also a pre-patched ROM circulating named `Seiken Densetsu 3 (Japan) [En by LNF+Neill Corlett+SoM2Freak v1.01]`. This is covered by rule 4 and 5.
 - The mere name `SEIKEN3` does not provide enough information to determine the language. Most likely it will be English. In any case, according to the above rule-set, the last rule will apply with English as fallback.
 
 ## UI Changes
@@ -76,17 +95,29 @@ Also, the program will check for unsupported characters on input, which means yo
 
 For compatibility both half-width and full-width characters are supported.
 
-A player's name is limited to a maximum length of 6 characters.
+A player's name is limited to a maximum length of 6 characters for the Japanese release and all translation patches based on it. Only the multilingual release from the Collection of Mana allows a maximum length of 12 characters.
+
+## Regarding the Nintendo Switch Release
+
+In 2017 the Seiken Densetsu Collection was released that had an identical copy of the original Seiken Densetsu 3 release. As such, this version uses the exact same encoding scheme, and it will be auto-detected as the original Japanese release.
+
+In 2019 the Collection of Mana was released internationally. For the first time, it offered official translations for English, French, German and Spanish, but neither Italian nor any other language if you so will ;) Besides a newly translated script, some character names, location and item names were also changed. Whichever translation you prefer, is up to you. Although, I have to admit that the German translation from G-Trans is pretty darn awesome, hehe.
+
+The biggest difference, however, is the character encoding. The multilingual version uses the same character set for all aforementioned languages, which is basically identical to Latin-1. Because it uses a single byte encoding, player names can be twice as long, allowing for up to 12 characters!
+
+**Experimental.** Because there are some encoding issues regarding quotation marks, an alternative translation patch option with a fix is offered. Normally, the single and the double quotation mark from the ASCII range appear as typographical ones in-game. However, the character set also does have the normal straight ones in its repertoire. The fix makes it so, that you can use both straight and typographical quotation marks alongside for your character names. So only the single and the double quotation marks from the ASCII range are transformed, and the typographical quotation marks from outside the ASCII range are unaffected from this fix.
 
 ## Remarks
 
-There are currently two encoding files to choose from when converting from Japanese to Unicode: One with full-width encoding in Unicode and the other one with half-width encoding in Unicode. Both work equally well. For now I went with full-width.
+There are currently two encoding files to choose from when converting from Japanese to Unicode: One with full-width encoding in Unicode and the other one with half-width encoding in Unicode. Both work equally well, and the only effect it has the visual presentation in the text input box. For now I went with full-width.
 
 You can change this by simply overwriting `encoding_japanese_to_unicode.json` with either `encoding_japanese_to_unicode_halfwidth.json` or `encoding_japanese_to_unicode_fullwidth.json`.
 
-Despite, when converting from Unicode to Japanese, both half-width and full-width characters are supported regardlessly.
+This said, when converting from Unicode to Japanese, both half-width and full-width characters are encoded properly.
 
 ## Known Issues
 
 - The Italian and the Spanish cartridge encoding both have a codepoint for the double L (ll). However, there is no according Unicode codepoint for it.
 
+- The multilingual encoding has multiple conflicting codepoints for quotation marks.
+

sd3save_editor/data/encoding_unicode_to_world.json

@@ -0,0 +1,266 @@
+{
+"\u0020":"\u0020",
+"\u0021":"\u0021",
+"\u0022":"\u0022",
+"\u0025":"\u0025",
+"\u0027":"\u0027",
+"\u0028":"\u0028",
+"\u0029":"\u0029",
+"\u002A":"\u002A",
+"\u002B":"\u002B",
+"\u002C":"\u002C",
+"\u002D":"\u002D",
+"\u002E":"\u002E",
+"\u002F":"\u002F",
+"\u0030":"\u0030",
+"\u0031":"\u0031",
+"\u0032":"\u0032",
+"\u0033":"\u0033",
+"\u0034":"\u0034",
+"\u0035":"\u0035",
+"\u0036":"\u0036",
+"\u0037":"\u0037",
+"\u0038":"\u0038",
+"\u0039":"\u0039",
+"\u003A":"\u003A",
+"\u003C":"\u003C",
+"\u003D":"\u003D",
+"\u003E":"\u003E",
+"\u003F":"\u003F",
+"\u0041":"\u0041",
+"\u0042":"\u0042",
+"\u0043":"\u0043",
+"\u0044":"\u0044",
+"\u0045":"\u0045",
+"\u0046":"\u0046",
+"\u0047":"\u0047",
+"\u0048":"\u0048",
+"\u0049":"\u0049",
+"\u004A":"\u004A",
+"\u004B":"\u004B",
+"\u004C":"\u004C",
+"\u004D":"\u004D",
+"\u004E":"\u004E",
+"\u004F":"\u004F",
+"\u0050":"\u0050",
+"\u0051":"\u0051",
+"\u0052":"\u0052",
+"\u0053":"\u0053",
+"\u0054":"\u0054",
+"\u0055":"\u0055",
+"\u0056":"\u0056",
+"\u0057":"\u0057",
+"\u0058":"\u0058",
+"\u0059":"\u0059",
+"\u005A":"\u005A",
+"\u005B":"\u005B",
+"\u005D":"\u005D",
+"\u005F":"\u005F",
+"\u0061":"\u0061",
+"\u0062":"\u0062",
+"\u0063":"\u0063",
+"\u0064":"\u0064",
+"\u0065":"\u0065",
+"\u0066":"\u0066",
+"\u0067":"\u0067",
+"\u0068":"\u0068",
+"\u0069":"\u0069",
+"\u006A":"\u006A",
+"\u006B":"\u006B",
+"\u006C":"\u006C",
+"\u006D":"\u006D",
+"\u006E":"\u006E",
+"\u006F":"\u006F",
+"\u0070":"\u0070",
+"\u0071":"\u0071",
+"\u0072":"\u0072",
+"\u0073":"\u0073",
+"\u0074":"\u0074",
+"\u0075":"\u0075",
+"\u0076":"\u0076",
+"\u0077":"\u0077",
+"\u0078":"\u0078",
+"\u0079":"\u0079",
+"\u007A":"\u007A",
+
+"\u3000":"\u0020",
+"\uFF01":"\u0021",
+"\uFF02":"\u0022",
+"\uFF05":"\u0025",
+"\uFF07":"\u0027",
+"\uFF08":"\u0028",
+"\uFF09":"\u0029",
+"\uFF0A":"\u002A",
+"\uFF0B":"\u002B",
+"\uFF0C":"\u002C",
+"\uFF0D":"\u002D",
+"\uFF0E":"\u002E",
+"\uFF0F":"\u002F",
+"\uFF10":"\u0030",
+"\uFF11":"\u0031",
+"\uFF12":"\u0032",
+"\uFF13":"\u0033",
+"\uFF14":"\u0034",
+"\uFF15":"\u0035",
+"\uFF16":"\u0036",
+"\uFF17":"\u0037",
+"\uFF18":"\u0038",
+"\uFF19":"\u0039",
+"\uFF1A":"\u003A",
+"\uFF1C":"\u003C",
+"\uFF1D":"\u003D",
+"\uFF1E":"\u003E",
+"\uFF1F":"\u003F",
+"\uFF21":"\u0041",
+"\uFF22":"\u0042",
+"\uFF23":"\u0043",
+"\uFF24":"\u0044",
+"\uFF25":"\u0045",
+"\uFF26":"\u0046",
+"\uFF27":"\u0047",
+"\uFF28":"\u0048",
+"\uFF29":"\u0049",
+"\uFF2A":"\u004A",
+"\uFF2B":"\u004B",
+"\uFF2C":"\u004C",
+"\uFF2D":"\u004D",
+"\uFF2E":"\u004E",
+"\uFF2F":"\u004F",
+"\uFF30":"\u0050",
+"\uFF31":"\u0051",
+"\uFF32":"\u0052",
+"\uFF33":"\u0053",
+"\uFF34":"\u0054",
+"\uFF35":"\u0055",
+"\uFF36":"\u0056",
+"\uFF37":"\u0057",
+"\uFF38":"\u0058",
+"\uFF39":"\u0059",
+"\uFF3A":"\u005A",
+"\uFF3B":"\u005B",
+"\uFF3D":"\u005D",
+"\uFF3F":"\u005F",
+"\uFF41":"\u0061",
+"\uFF42":"\u0062",
+"\uFF43":"\u0063",
+"\uFF44":"\u0064",
+"\uFF45":"\u0065",
+"\uFF46":"\u0066",
+"\uFF47":"\u0067",
+"\uFF48":"\u0068",
+"\uFF49":"\u0069",
+"\uFF4A":"\u006A",
+"\uFF4B":"\u006B",
+"\uFF4C":"\u006C",
+"\uFF4D":"\u006D",
+"\uFF4E":"\u006E",
+"\uFF4F":"\u006F",
+"\uFF50":"\u0070",
+"\uFF51":"\u0071",
+"\uFF52":"\u0072",
+"\uFF53":"\u0073",
+"\uFF54":"\u0074",
+"\uFF55":"\u0075",
+"\uFF56":"\u0076",
+"\uFF57":"\u0077",
+"\uFF58":"\u0078",
+"\uFF59":"\u0079",
+"\uFF5A":"\u007A",
+
+"\u00A0":"\u00A0",
+"\u00A1":"\u00A1",
+"\u00A9":"\u00A9",
+"\u00AB":"\u00AB",
+"\u00B0":"\u00B0",
+"\u00B9":"\u00B9",
+"\u00BB":"\u00BB",
+"\u00BF":"\u00BF",
+"\u00C0":"\u00C0",
+"\u00C1":"\u00C1",
+"\u00C2":"\u00C2",
+"\u00C4":"\u00C4",
+"\u00C6":"\u00C6",
+"\u00C7":"\u00C7",
+"\u00C8":"\u00C8",
+"\u00C9":"\u00C9",
+"\u00CA":"\u00CA",
+"\u00CB":"\u00CB",
+"\u00CC":"\u00CC",
+"\u00CD":"\u00CD",
+"\u00CE":"\u00CE",
+"\u00CF":"\u00CF",
+"\u00D1":"\u00D1",
+"\u00D2":"\u00D2",
+"\u00D3":"\u00D3",
+"\u00D4":"\u00D4",
+"\u00D5":"\u00D5",
+"\u00D6":"\u00D6",
+"\u00D7":"\u00D7",
+"\u00D9":"\u00D9",
+"\u00DA":"\u00DA",
+"\u00DB":"\u00DB",
+"\u00DC":"\u00DC",
+"\u00DF":"\u00DF",
+"\u00E0":"\u00E0",
+"\u00E1":"\u00E1",
+"\u00E2":"\u00E2",
+"\u00E4":"\u00E4",
+"\u00E6":"\u00E6",
+"\u00E7":"\u00E7",
+"\u00E8":"\u00E8",
+"\u00E9":"\u00E9",
+"\u00EA":"\u00EA",
+"\u00EB":"\u00EB",
+"\u00EC":"\u00EC",
+"\u00ED":"\u00ED",
+"\u00EE":"\u00EE",
+"\u00EF":"\u00EF",
+"\u00F1":"\u00F1",
+"\u00F2":"\u00F2",
+"\u00F3":"\u00F3",
+"\u00F4":"\u00F4",
+"\u00F5":"\u00F5",
+"\u00F6":"\u00F6",
+"\u00F7":"\u00F7",
+"\u00F9":"\u00F9",
+"\u00FA":"\u00FA",
+"\u00FB":"\u00FB",
+"\u00FC":"\u00FC",
+
+"\u25BC":"\u003B",
+"\u0026":"\u0040",
+"\u2026":"\u005C",
+"\u0023":"\u005E",
+"\u007D":"\u0060",
+"\u2212":"\u007D",
+"\u2122":"\u0099",
+"\u2018":"\u00A3",
+"\u2019":"\u00A4",
+"\u201C":"\u00A5",
+"\u201D":"\u00A6",
+"\u201E":"\u00AA",
+"\u2190":"\u00AC",
+"\u2B05":"\u00AC",
+"\uFFE9":"\u00AC",
+"\uD83E\uDC44":"\u00AC",
+"\u2191":"\u00AD",
+"\u2B06":"\u00AD",
+"\uFFEA":"\u00AD",
+"\uD83E\uDC45":"\u00AD",
+"\u2192":"\u00AE",
+"\u2B95":"\u00AE",
+"\uFFEB":"\u00AE",
+"\uD83E\uDC46":"\u00AE",
+"\u2193":"\u00AF",
+"\u2B07":"\u00AF",
+"\uFFEC":"\u00AF",
+"\uD83E\uDC47":"\u00AF",
+"\u2010":"\u00B1",
+"\u2013":"\u00B2",
+"\u2014":"\u00B3",
+"\u2070":"\u00BA",
+"\u0152":"\u00D8",
+"\u0153":"\u00F8",
+"\u1E9E":"\u00FF"
+}
+