Hi Jonathan,
As Mendix uses Postrgres by default, the docs there give you 4,294,967,295 rows. (source)
However this post gives you an overview of when it would start to slow down performance.
We have currently a table in one of our apps that has over 80M records stored, an unfiltered page with a datagrid takes about 10 seconds to load.
Good luck with making a decision I hope this information is valuable!
KR, Jurre