Thai font collapse when using more than 1 tone marks

[hackinteach]

Describe the bug

According to this documentation, adding font with Unicode support works with Thai language as well. Unfortunately, when using more elaborate font other than Waree from the example, the tone marks collapse when having 2 marks in 1 words, for example: ที่ นั้น นี่ ทั้ง มั้ง บุหรี่

Minimal code
Please include some minimal Python code reproducing your issue:

from glob import glob

pdf = FPDF()
fonts = glob("fonts/*.ttf")

label = u"ที่ นั้น นี่ ทั้ง มั้ง บุหรี่"
pdf.set_text_color(0,0,0)
pdf.set_font_size(20)
pdf.add_page()
y = 10
for f in fonts:
    pdf.add_font(family=f, fname=f, uni=True)
    pdf.set_font(f)
    pdf.set_xy(20,y)
    pdf.cell(w=pdf.get_string_width(f), h=10, txt=f"{f} - {label}")
    y += 20
pdf.output("test font.pdf")

Generated sample shown below, Waree (the working font) is in the green box:

Environment

• OS 12.4 Monterey
• Python version 3.8.12
• fpdf2 version used 2.5.5

[Lucas-C]

Thank you for reporting this.

This looks like a duplicate of #365 and maybe #458

There is currently an effort to change our font-parsing code: #418
Hopefully, this may improve the support for Thai language on the long term.

Of course, any other suggestion or contribution to improve the situation is welcome 😊

[gmischler]

This looks like a duplicate of #365

Makes you wonder why it comes out correctly with one font...
These don't seem to be ligatures, but rather accent characters, which fpdf2 supports perfectly fine.

Note that in the example, the "more elaborate" fonts all show the characters in the same bland shape, with slightly varying thickness. Now look at how Prompt Thin should really look like:

This is clearly not what we're seeing here.

Just being a font with "Unicode support" does not garantee for any range of characters to actually be present in a font file (otherwise, every Unicode font would be 20+ MB large). You'll have to check which languages resp. writing systems it actually supports. In the case of Prompt Thin, support is indicated for "latin" only.

Indeed, the rendered text looks mostly like latin characters. Does fpdf2 actually do some substitution for non-existing glyphs? Or do the/some PDF viewers do that?

Solution for @hackinteach: Find and use fonts that actually include glyph data for the Thai script, and everything will be fine. 🇹🇭 👍

[hackinteach]

@gmischler Thank you for your suggestion. I did some more finding on Thai fonts and found the ones working now (I should have go deeper link in the Unicode documentation).

Might be useful for someone in the future, I download the fonts from Fonts-TLWG using this link and the results are as follow:

[Lucas-C]

Great, thank you for your answer @gmischler!

@hackinteach / @gmischler : would you like to contribute a PR on docs/Unicode.md to add a mention of this? 😊
Maybe simply a new paragraph about Thai language, and how to select fonts to properly display text in this language?

[hackinteach]

Sure, my pleasure :)

[gmischler]

Maybe simply a new paragraph about Thai language, and how to select fonts to properly display text in this language?

I don't think this is specific to the Thai language.
We can of course use that one as an example, but the issue is really about all non-latin writing systems.

[Lucas-C]

FYI @hackinteach, @andersonhc PR #820 has been merged today.

Thai text is now better supported!
The documentation is there: https://pyfpdf.github.io/fpdf2/TextShaping.html

To test it, you can install fpdf2 directly from the master branch of this repo with this command:

pip install git+https://github.com/PyFPDF/fpdf2.git@master