No, this is not possible because the model would need to know what part of a response might contain this information. It could be anything.
And do note that an attacker would need to be able to set the log level to trace and capture the logs. In that case the attacker is already in and can do a lot more damage.
Regards,
Ronald