I also ran into this question where I was able to show an image in 404.html, but using the same setup would not display the image in offline.html when the app is down.
What you can do is turn the image into a base64 string and use it within the html directly like this:
<img src="data:image/png;base64,ABC..." />
Have a look at this post. This is about using images from Sass but you may try something similar with your html file. Place images in the mentioned folders and provide its path.
https://forum.mendix.com/link/questions/104030
It is possible to display images in custom error pages. Make sure the images are placed in the error_page folder.
And make sure that the img tag uses url like /error_page/image.jpg. For example:
<img src="/error_page/logo.svg">
See also: https://docs.mendix.com/howto/front-end/custom-error-page/
For Css styling : <link rel="stylesheet" type="text/css" href="/error_page/offline.css">
for Images :<img src="/error_page/rds-toolbox-dark.svg" alt="503 Error">
You need to add path /error_page/ before css or images, This should fix it