Hello there :),
in the request tab in your rest service, go for binary for the entire request. Then in Content, select your object of the fileDocument and use Contents attribute. For example ->
Contents are always in binary and exactly what you need.