What you want to do is take the time of each of these dates and put them on the same date.
So for instance:
minutesBetween
(
parseDateTime('01-01-2020 ' + formatDateTime($Date1, ‘HH:mm:ss’), 'dd-MM-yyyy HH:mm:ss'),
parseDateTime('01-01-2020 ' + formatDateTime($Date2, ‘HH:mm:ss’), 'dd-MM-yyyy HH:mm:ss')
)