Seems that the underline button only shows up when using the full toolbar and is ignored when using a custom setup.
Easy to reproduce so I would suggest creating a ticket for this with support as this is a widget published by Mendix themselves.
Version 2.1.2 of the Rich Text widget have been released by Mendix to fix this issue.
I just downloaded the new widget version, it did not fix the problem.
I tried deleting the widget from my page and adding it again, nothing.
Result after using the latest version of RichTextEditor.